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Executive Summary 



In the rapidly changing world of the 21st century, science literacy is an essential goal for 
all of our nation’s youth. Through science education, children come to understand the 
world in which they live and learn to apply scientific principles in many facets of their 
lives. In addition, our country has an obligation to provide young people who choose to 
pursue careers in science and technology with a strong foundation for their postsecondary 
study and work experience. The nation’s future depends on scientifically literate citizens 
who can participate as informed members of society and as a highly skilled scientific 
workforce, well prepared to address challenging issues at the local, national, and global 
levels. Recent studies, including national and international assessments, indicate that our 
schools still do not adequately educate all students in science. 

Science seeks to increase our understanding of the natural world through empirical evi- 
dence. Such evidence gathered through observation and measurement allows for an ex- 
planation and prediction of natural phenomena. Hence, a scientifically literate person is 
familiar with the natural world and understands key facts, concepts, principles, laws, and 
theories of science, such as the motion of objects, the function of cells in living organ- 
isms, and the properties of Earth materials. Further, a scientifically literate person can 
connect ideas across disciplines; for example, the conservation of energy in physical, life, 
Earth, and space systems. Scientific literacy also encompasses understanding the use of 
scientific principles and ways of thinking to advance our knowledge of the natural world 
as well as the use of science to solve problems in real-world contexts, which this docu- 
ment refers to as “Using Technological Design.” 

The National Assessment of Educational Progress (NAEP) and its reports are a key 
measure in informing the nation on how well the goal of scientific literacy for all students 
is being met. The Science Framework for the 2009 National Assessment of Educational 
Progress sets forth recommendations for the design of a new science assessment. The 
assessment resulting from this framework will start a new NAEP science trend (i.e., 
measure of student progress in science) beginning in 2009. This framework represents a 
unique opportunity to build on previous NAEP science work as well as key developments 
in science standards, assessments, and research. This document is intended to inform the 
general public, educators, policymakers, and others about what students are expected to 
know and be able to do in science as part of The Nation’s Report Card, a program of the 
U.S. Department of Education (ED) that reports on NAEP findings. 

In 1988, Congress created the National Assessment Governing Board to set policy for 
NAEP. The National Center for Education Statistics (NCES, a division of ED) carries out 
NAEP. As the ongoing national indicator of the academic achievement of U.S. students, 
NAEP regularly collects information on representative samples of students in grades 4, 8, 
and 12 and periodically reports on student achievement in reading, mathematics, writing, 
science, and other subject areas. NAEP scores are always reported at the aggregate level, 
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not for individual students or schools. (By law, NAEP cannot report results for individual 
students.) For science, NAEP results are reported at the national, state, and select district 
levels. The district reports are provided for urban school systems that volunteer for the 
Trial Urban District component of NAEP. 

NAEP produces comparative student achievement results according to demographic 
factors such as gender, race/ethnicity, and geographic region. Results are also provided in 
terms of student, teacher, and school background variables related to science achieve- 
ment. Taken together, this information from NAEP helps the general public, educators, 
and policymakers make informed decisions about education. Interested individuals can 
access performance results and released questions through NAEP reports and Web sites. 

A new framework to guide the science assessment in 2009 is necessary for several 
reasons: publication of National Standards for science literacy, advances in both science 
and cognitive research, growth in national and international science assessments, and 
increases in innovative assessment approaches. This 2009 framework presents the content 
to be assessed as well as the conceptual base for the assessment. It is intended for a 
general audience. A more detailed technical document, Science Assessment and Item 
Specifications for the 2009 NAEP, is a companion piece. The specifications document 
provides detailed information on the content to be assessed, item development, and other 
aspects of assessment development and administration. The audience for the science 
Specifications is NCES and the NAEP assessment development contractor. 

Key Features 



This framework is the result of extraordinary effort and commitment by hundreds of in- 
dividuals across the country, including some of the nation’s leading scientists, science 
educators, policymakers, and assessment experts. Under contract to the Governing Board, 
WestEd and the Council of Chief State School Officers (CCSSO) spent 18 months devel- 
oping the framework; this process involved committees, regional hearings, and other pub- 
lic forums. The Governing Board also engaged an external review panel to evaluate the 
draft framework and convened a public hearing to gather additional input during the 
development process. 

The new framework incorporates the following key features: 

• Its design is based on widely accepted national science standards and assessments 
in addition to state curriculum standards. However, it is intended to inform devel- 
opment of an assessment, not to advocate for a particular approach to instruction 
or to represent the entire range of science content and skills. 

• In view of the need to keep the United States and its youth internationally com- 
petitive in science and technology, the framework development process gave spe- 
cial consideration to international assessment frameworks, such as those for 
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Trends in International Mathematics and Science Study (TIMSS) and Programme 
for International Student Assessment (PISA). 

• The breadth of the science principles represented in the source materials made it 
necessary to focus on the foundational and pervasive knowledge within each dis- 
cipline and to pare down the science content to be assessed. 

• The framework is based on scientific knowledge and processes derived from 
tested explanations and supported by accumulated empirical evidence. Explana- 
tions of natural phenomena that rely on nonscientific views are not reflected in the 
framework. 

• Science content is presented in detailed, grade- specific charts that also allow the 
reader to see the progression in complexity of ideas across grades. 

• Every attempt has been made to be free of error in describing the science content. 
The language used strives to be accurate but not technical so as to make the 
framework accessible to a wide audience. 

• The focus is on students’ conceptual understanding, that is, their knowledge and 
use of science facts, concepts, principles, laws, and theories. Students’ abilities to 
engage in some components of scientific inquiry and technological design are also 
reflected in the framework. 

• New types of items are recommended, including the use of interactive computer 
tasks. 

Content of the Framework 



The framework describes the science content and science practices that form the basis for 
the 2009 NAEP Science Assessment. It also discusses item distribution and types of 
items, as well as draft achievement levels. Finally, it recommends several small-scale, 
special studies to be considered in conjunction with the 2009 or future science assess- 
ments. 

SCIENCE CONTENT 

The science content for the 2009 NAEP is defined by a series of statements that describe 
key facts, concepts, principles, laws, and theories in three broad areas: 

• Physical Science 

• Life Science 

• Earth and Space Sciences 
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Physical Science deals with matter, energy, and motion; Life Science deals with struc- 
tures and functions of living systems and changes in living systems; and Earth and Space 
Sciences deal with Earth in space and time, Earth structures, and Earth systems. Details 
about the science content and the science content statements can be found in chapter two. 

SCIENCE PRA CTICES 

The second dimension of the framework is defined by four science practices: 

• Identifying Science Principles 

• Using Science Principles 

• Using Scientific Inquiry 

• Using Technological Design 

These practices can be combined with any science content statement to generate student 
performance expectations, and assessment items can then be developed based on these 
performance expectations. The cognitive demands placed on students as they engage in 
assessment tasks are also described. The science practices and cognitive demands are 
explained more fully in chapter three. 

DISTRIBUTION OF ITEMS 

As measured by student response time, the distribution of items by content area should be 
as follows: approximately equal across Physical Science, Life Science, and Earth and 
Space Sciences at grade 4; more emphasis on Earth and Space Sciences at grade 8; and a 
shift to more emphasis on Physical Science and Life Science at grade 12. With respect to 
science practices, at all grades the greatest emphasis should be on identifying and using 
science principles and slightly less than a third of the time should be spent on items 
related to using scientific inquiry. Specific recommended percentages are discussed in 
chapter four. 

TYPES OF ITEMS 

Item types for the 2009 NAEP Science Assessment fall into two broad categories: 
selected-response items (such as multiple choice) and constructed-response items (such 
as short answer). As measured by student response time, 50 percent of the assessment 
items at each grade level should be selected response and 50 percent should be con- 
structed response. To further probe students’ abilities to combine their understanding 
with the investigative skills that reflect practices, a subset of the students sampled should 
receive an additional 30 minutes to complete hands-on performance or interactive com- 
puter tasks. At each grade, at least four of these tasks should be included. Of these four 
tasks, there should be at least one hands-on performance and one interactive computer 
task; the number of interactive computer tasks should not exceed the number of hands-on 
performance tasks. Chapter four contains more information about types of items. 
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Hands-on Performance Tasks 



In hands-on performance tasks, students manipulate selected physical objects and try to 
solve a scientific problem involving the objects. NAEP hands-on performance tasks 
should provide students with a concrete task (problem) along with equipment and 
materials. Students should be given the opportunity to determine scientifically justifiable 
procedures for arriving at a solution. Students’ scores should be based on both the solu- 
tion and the procedures created for carrying out the investigation. Further discussion 
about hands-on performance tasks can be found in chapter four. 

Interactive Computer Tasks 

There are four types of interactive computer tasks: (1) information search and analysis, 

(2) empirical investigation, (3) simulation, and (4) concept maps. Information search and 
analysis items pose a scientific problem and ask students to query an information data- 
base and analyze relevant data to address the problem. Empirical investigation items 
place hands-on performance tasks on the computer and invite students to design and con- 
duct a study to draw conclusions about a problem. Simulation items model systems (e.g., 
food webs) and ask students to manipulate variables, and predict and explain resulting 
changes in the system. Concept map items probe aspects of the structure or organization 
of students’ scientific knowledge by providing concept terms and having students create a 
logical graphic organizer. Chapter four contains more information about interactive com- 
puter tasks. 

Special Studies 



On occasion, NAEP carries out special studies to inform future assessments. Three spe- 
cial study proposals ranked as top priority for the NAEP Science Assessment: 

• “Exchangeability” of hands-on performance and interactive computer investiga- 
tions 

• Impact of variation in item format and language demand on the performance of 
English language learners and students with disabilities 

• Computer adaptive testing to assess the development of student understanding of 
Earth systems 

Details of these proposed studies are found in chapter four. 
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Achievement Levels 



Results of the NAEP science assessment are reported as average scores for groups of 
students and as percentages of students who attain the Basic, Proficient, or Advanced 
achievement levels. Preliminary descriptions of these achievement levels are contained in 
appendix B . 
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The Science Framework for the 2009 National Assessment of Educational Progress is the 
result of extraordinary effort and commitment by hundreds of individuals across the 
country, including some of the nation’s leading scientists, science educators, policy- 
makers, and assessment experts. Every attempt has been made to be accurate in the 
description and representation of the science content. Existing framework development 
policies and procedures ensure periodic reviews and revisions, if necessary, to reflect 
advancements in scientific knowledge. Because this is a public endeavor, the Governing 
Board (www.nagb.org) welcomes suggestions for future versions of the framework. 
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• Online survey with a total of 1,769 responses, representing all 50 states and the 
District of Columbia 

Smithsonian Institution 

• Presentation to education staff 

Triangle Coalition for Science and Technology Education 

• Annual conference 

U.S. -China Education Leaders Forum on Math and Science Education 



Testimony at Governing Board Public Hearing, October 2005, Washington, DC 



Achieve (represented by Jean Slattery) 

American Association for the Advancement of Science (represented by Alan Leshner) 
Intel Foundation (represented by Wendy Hawkins) 

International Technology Education Association (represented by Kendall Starkweather) 
Museum of Science, Boston (represented by Patti Curtis) 

National Academy of Engineering (represented by William Wulf) 

National Science Teachers Association (represented by Gerald Wheeler) 

Thomas B. Fordham Foundation (represented by Michael Petrilli) 



Governing Board Invitational Science and Industry Discussion Forum, May 2005, 
Atlanta, GA 



Lela Blackburn, Atlanta Public Schools 

Pete Briggs, Scientific Atlanta 

Robert DeHaan, Emory University 

Victor De Jesus, Georgia Tech Research Institute 

Maynard Hammond, Scientific Atlanta 

Stewart Holm, Georgia Pacific 

Lamar May, Lockheed Martin (retired) 

Seth Marder, Georgia Tech 

Tom McKlin, Center for Education Integrating Science, Mathematics, and Computing 
(CEISMC) 

Chris Miller, Cytec Surface Specialties 

Bob Moore, Southern Regional Education Board 

Paul Ohme, CEISMC 

Rick Paulus, EMS Technologies 

Randy Rowe, GE Energy 

Ken Stefanini, Georgia Power 

Marion Usselman, CEISMC 

Donna Whiting, CEISMC 
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In addition, the Governing Board received feedback on the draft framework via hundreds 
of individual letters and e-mails. The authors of these letters and e-mails included scien- 
tists; teachers; university administrators and professors; education researchers; curricu- 
lum developers; assessment and measurement specialists; and representatives from 
science industry, museums and science centers, community interest groups, professional 
organizations, and government agencies at the federal, state, and district levels. 
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Chapter One 



Overview 

NAEP measures student science achievement nationally, state by state, and, most re- 
cently, across selected urban school districts. Periodically, the framework underlying the 
science assessment is revised or updated. This document, Science Framework for the 
2009 National Assessment of Educational Progress, contains a new set of recommenda- 
tions for the NAEP Science Assessment to be administered in 2009 and beyond. The 
framework provides guidance on the science content to be assessed, the types of assess- 
ment questions, and the administration of the assessment. 

For more than 35 years, NAEP has gathered information on student achievement in se- 
lected academic subjects. Originally, assessments were age-based samples of 9-, 13-, and 
17-year-old students. Beginning in 1983, the assessment also has included grade -based 
samples of students in grades 4, 8, and 12. Currently, long-term trend NAEP continues to 
assess 9-, 13-, and 17-year-olds in mathematics and reading, while main NAEP assesses 
students in grades 4, 8, and 12. More information about differences between long-term 
trend and main NAEP can be found on the Internet at 
http://nces.ed.gov/nationsreportcard/about/ltt_main_diff.asp (NCES 2005b). 

NAEP has become an important source of information on what U.S. students know and 
are able to do in reading, mathematics, science, U.S. history, writing, and other subjects. 

In addition, NAEP provides information on how student performance has changed over 
time. Since the 1990s, in addition to the national-level assessments, NAEP has conducted 
and reported state-level assessments at grades 4 and 8 in reading, mathematics, writing, 
and science. State-level as well as national science assessments were conducted in 1996, 
2000, and 2005. The resulting data on student knowledge and performance have been 
accompanied by background information that allows analyses of a number of student 
demographic and instructional factors related to achievement. The assessments have been 
designed to allow comparisons of student performance over time and among subgroups 
of students according to region, parental education, gender, and race/ethnicity. In 2002, 
NAEP began a Trial Urban District Assessment (TUDA) in districts that volunteered to 
participate. TUDA has continued through 2005, when 10 districts took part in NAEP 
assessments that produced district-level results. 

Need for a New Framework 



The framework that guided the last three NAEP Science Assessments (administered in 
1996, 2000, and 2005) was developed some 15 years ago. Since then, the following 
developments have taken place, making it necessary to create a new framework for 
assessing science in 2009 and beyond: 
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• Publication (for the first time) of National Standards for science literacy in 

National Science Education Standards (National Research Council [NRC] 1996) 
(National Standards) and Benchmarks for Scientific Literacy (American Associa- 
tion for the Advancement of Science [AAAS] 1993) (Benchmarks). Since their 
publication, these two national documents have informed state science standards. 

• Advances in science research (e.g., on the relationship between human activity 
and the natural world) that have increased knowledge and, as a consequence, have 
influenced the school curriculum in the fields of physical, life, and Earth and 
Space Sciences. 

• Advances in cognitive research (e.g., on how students learn increasingly com- 
plex material over time) that have yielded new insights into how and what stu- 
dents learn about science (NRC 1999c). 

• Growth in the prevalence of science assessments nationally and internation- 
ally, including the requirements in the current federal education legislation (No 
Child Left Behind) for science assessment starting in 2007, the ongoing interna- 
tional assessment (TIMSS) (http://timss.bc.edu), and PISA (www.pisa.oecd.org). 

• Growth in innovative assessment approaches that probe students’ understand- 
ing of science in greater depth than before (e.g., clusters of items tapping stu- 
dents’ conceptions of the natural world), sometimes with the use of computer 
technology (NRC 2001). 

• Increased inclusion of formerly excluded groups in science assessments (e.g., 
students with disabilities and English language learners), requiring a new assess- 
ment to be as accessible as possible and also to incorporate accommodations so 
the widest possible range of students can be fairly assessed. Accommodations 
should not alter the science constructs being measured. 

Context for Planning the Framework 



Any NAEP framework must be guided by NAEP purposes as well as the policies and 
procedures of the Governing Board, which oversees NAEP. For the NAEP Science 
Assessment, the main purpose of the framework is to establish what students should 
know and be able to do in science for the 2009 and future assessments. Meeting this pur- 
pose requires a framework built around what communities involved in science and sci- 
ence education consider as a rigorous body of science knowledge and skills that are most 
important for NAEP to assess. 

In prioritizing the content, the framework developers used the NAEP Science Assessment 
steering committee guidelines (presented later in this chapter), which recommended the 
two national documents (National Standards and Benchmarks) as representative of the 
leading science communities and their expectations for what students should know and be 
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able to do in science. As curriculum frameworks, however, these documents cover a very 
wide range of science content and performance. The inclusive nature of both documents 
demonstrates the difficulty of identifying a key body of knowledge for students to learn 
in science and, therefore, what should be assessed. Neither document limits or prioritizes 
content as is necessary for developing an assessment, posing a considerable challenge to 
the framework developers. The development of the framework also was informed by re- 
search in science and science education, best practices, international assessment frame- 
works, and state standards. 

The Framework Development Process 



In September 2004, the Governing Board awarded a contract to WestEd and CCSSO to 
develop a recommended framework and specifications. WestEd and CCSSO, in collabo- 
ration with AAAS, the Council of State Science Supervisors (CSSS), and the National 
Science Teachers Association (NSTA), used a process designed to accomplish the pur- 
poses of this project with special attention given to the assessment issues specific to K-12 
science achievement. The process for developing the framework and related products was 
inclusive and deliberate, designed to achieve as much broad-based input as possible. 

A two-tiered committee structure, consisting of a steering committee and a planning 
committee, provided the expertise to develop the framework as specified by the Govern- 
ing Board. (See “NAEP Science Project Staff and Committees” for lists of committee 
members.) The two committees were composed of members who were diverse in terms 
of role, gender, race/ethnicity, region of the country, perspective, and expertise regarding 
the content of the assessment to be developed. 

The science assessment steering committee included leaders in science, science educa- 
tion, general education, assessment, and various public constituencies and set the course 
for the project. Functioning as a policy and oversight body, this group developed a charge 
that outlined the planning committee’s responsibilities in developing the framework. The 
committee also reviewed and provided feedback on drafts of the framework and related 
materials. 

The science assessment planning committee, supported by the project staff, was the 
development and production group responsible for drafting the framework, the Specifica- 
tions, recommendations for background variables, designs for one or more small-scale 
studies, and preliminary science achievement level descriptions. This committee was 
made up of science teachers, district and state science personnel, science educators in 
higher education, scientists, and assessment experts. The planning committee’s work was 
guided by policies, goals, and principles identified by the steering committee. In addition, 
the planning committee used a number of resources to facilitate its work. These resources 
included an issues and recommendations paper (Champagne et al. 2004) developed spe- 
cifically for this NAEP project; the frameworks and specifications for the 2005 NAEP 
Mathematics Assessment and 1996-2005 NAEP Science Assessments; other NAEP 
reports and documents produced by the Governing Board and NCES; international 



NAEP 2009 Science Framework 



3 





assessment frameworks; syntheses of state and national curriculum standards; and re- 
search papers and resources provided by steering and planning committee members and 
project staff. 

The structure for conducting the work consisted of a series of meetings. From December 
2004 through September 2005, the steering committee met three times and the planning 
committee met six times; two of the steering committee meetings overlapped with plan- 
ning committee meetings. Governing Board staff supported and participated in the work 
of the committees during the meetings. In addition, between formal work sessions, Gov- 
erning Board members and staff provided ongoing feedback and guidance on project 
documents and processes. 

During spring 2005, CCS SO led a series of outreach efforts to solicit feedback on draft 
versions of the framework. Formal activities included the following: 

• A series of 13 regional meetings held across the country and hosted by CCSSO 
and CSSS members 

• A national meeting of CSSS representatives 

• A Web-based survey of science teachers distributed through NSTA 

• An invitational science and industry feedback forum held in Atlanta, GA, in con- 
junction with a Governing Board meeting 

These activities are discussed in A Summary of National Feedback Provided on Prelimi- 
nary Drafts Gathered from Surveys and Regional and National Feedback Meetings 
(CCSSO 2005). Feedback from these sessions has been incorporated into the framework. 
Examples include reducing the number of statements of science content to be assessed; 
comparing the old and new science frameworks; and ensuring a high level of consistency 
in scope, specificity, language, and format across the science content areas. 

Other related outreach activities included but were not limited to presentations and ses- 
sions held with AAAS, the CSSS annual conference; the NSTA national and regional 
conventions; meetings of the NRC’s Board on Science Education and Committee on 
Science Learning K-8; and CCSSO’s Mega-SCASS (State Collaborative on Assessment 
and Student Standards) conference, large-scale assessment conference, and Education 
Information Management Advisory Consortium (EIMAC). The Governing Board en- 
gaged an external review panel to evaluate the draft framework and convened a public 
hearing to gather additional input during the development process. (See “NAEP Science 
Project Staff and Committees” for more complete lists of individuals and organizations 
that contributed to the development of this framework.) The planning committee re- 
viewed feedback from these groups as well as that from the steering committee and made 
changes as it deemed appropriate. After final approval from the steering committee, the 
framework, specifications, and related products were submitted to the Governing Board 
for approval. The Governing Board unanimously approved the 2009 Science Framework 
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on November 18, 2005. Final copies of the specifications and related products were sub- 
mitted to the Governing Board in March 2006. 

Steering Committee Guidelines 



Because the steering committee guidelines were important to the planning committee in 
developing the framework, the major points of the guidelines are summarized below. 

(The complete document is contained in appendix A.) The guidelines consist of criteria 
that the framework and specifications must meet: 

• The framework is informed by the National Standards and Benchmarks . The 
framework should reflect the nation’s best thinking about the importance and age- 
appropriateness of science principles and thus be informed by two national docu- 
ments that were subject to extensive internal and external reviews during their 
development. 

• The framework reflects the nature and practice of science. The National Stan- 
dards and Benchmarks include standards that address science as inquiry, nature of 
science, history of science, and the manmade world. The framework should em- 
phasize the importance of these aspects of science education and should include 
the expectation that students will understand the nature and practice of science. 
Because the scientific disciplines are no longer practiced in isolation and research 
that cuts across discipline boundaries is common, the framework should identify 
some of the science concepts and skills that cut across the assessed content areas. 
The framework should address science in both the natural and manmade world, as 
well as social and historical contexts. 

• The framework uses assessment content, formats, and accommodations con- 
sistent with the objectives being assessed. It should be guided by the best avail- 
able research on assessment item design and delivery. The framework should in- 
clude student diversity as reflected in gender, geographic location, language pro- 
ficiency, race/ethnicity, socioeconomic status, and disability. The assessment 
should be designed and written to be accessible by the majority of students, 
minimizing the need for special accommodations for both students with disabili- 
ties and English language learners. However, students with special needs should 
be provided appropriate accommodations to allow them to participate in the as- 
sessment. The framework should reflect knowledge about the acquisition of key 
science concepts over time, based on research about how students learn. Critical 
content and skills should be articulated and assessed across grades 4, 8, and 12 
(vertically), as well as across the fields of science (horizontally) by creating items 
that are deliberately layered to achieve these goals. 

• The framework uses a variety of assessment formats. This includes well- 
constructed multiple-choice and open-ended items as well as performance tasks. 

In addition, multiple methods of assessment delivery should be considered, in- 
cluding appropriate uses of computer technology. 
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• Each achievement level — Basic, Proficient, and Advanced — includes a range 
of items assessing various levels of cognitive knowledge that is broad enough 
to ensure each knowledge level is measured with the same degree of 
accuracy. Descriptions of Basic, Proficient, and Advanced must be as clear as 
possible. 

• Connections among the framework, the specifications, and the assessment 
items themselves are transparent, coherent, and have a consistent level of 
specificity. The specifications should be written with detail consistent with the 
framework. The content addressed in the specifications should reflect the stan- 
dards and focus on the significant information and knowledge that students should 
retain over time (e.g., big ideas, fundamental understandings). The verbs used in 
the specifications should describe the expected target for assessment (e.g., iden- 
tify, describe, evaluate, relate, analyze, and demonstrate). The content expecta- 
tions should match in level of specificity and scope across the disciplines. The 
specifications should follow the idea of learning progressions. To assess overarch- 
ing concepts or themes, the specifications should reflect a layered understanding 
of growth in knowledge of the concepts. 

• The framework addresses the use of assessment data to conduct research on 
science learning and to improve science achievement. Data (background vari- 
ables) from the assessment should be collected in a way that provides details 
about the characteristics of the students being assessed (e.g., race/ethnicity and 
gender), the academic preparation of their teachers, and the nature of their 
schools. Such data provide feedback to educators for improving science instruc- 
tion and learning. 

Uses of NAEP Data 



For more than four decades, NAEP has provided information integral to reporting on the 
condition and progress of education at grades 4, 8, and 12 for the nation and, more re- 
cently, for the states and for a set of large, urban school districts. Legislation concerning 
NAEP states that its purpose is to provide, in a timely manner, a fair and accurate meas- 
urement of student academic achievement and reporting of trends in such achievement in 
reading, mathematics, and other subject matter (Public Law 107-279). 

Because of its rigorous design and methodology, NAEP reports are increasingly used for 
monitoring the state of education in the subjects that are assessed, as models for design- 
ing other large-scale assessments, and for secondary research purposes. 

MONITORING 

As the nation’s only ongoing survey of students’ educational progress, NAEP has 
become an increasingly important resource for obtaining information on what students 
know and can do. Because the information it generates is available to policymakers, 
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educators, and the public, NAEP can be used as a tool for monitoring student 
achievement in reading, mathematics, science, and other subjects at the national, state, 
and selected district levels. For example, NAEP reports (known as The Nation’s Report 
Card) compare student performance in a given subject across states, within the subject 
over time, or among groups of students within the same grade. NAEP also reports long- 
term achievement trends for 9-, 13-, and 17-year-olds in reading and mathematics (e.g., 
Perie, Moran, and Lutkus 2005). To the extent that individual state standards reflect the 
common core of knowledge and skills specified in the framework, state comparisons can 
legitimately be made. If a state has unique standards, any comparison is limited by the 
degree of mismatch between NAEP content and state content. Even with this caveat, 
NAEP still stands as a key indicator of what students know and can do in science at 
grades 4, 8, and 12. 

MODEL OF ASSESSMENT DEVELOPMENT AND METHODS 

NAEP assessment frameworks and specifications documents are used as resources for 
international, state, and local curriculums and assessments. The broad-based process used 
in the development of the frameworks and specifications means that current thinking and 
research are reflected in the descriptions of what students should know and be able to do 
in a given subject. In addition, NAEP uses a rigorous and carefully designed process in 
developing the assessment instruments. Pilot tests and internal and external reviews 
ensure that NAEP assessments are reliable and valid with respect to what they attempt to 
accomplish. This sophisticated methodology serves as a model for other assessment 
developers. Given the No Child Left Behind requirement to assess students in science, 
states may wish to use NAEP as one model to guide their own assessment development. 

RESEARCH AND POLLCY 

NAEP data include subject-matter achievement results (reported as both scale scores and 
achievement levels) for various subgroups; background information about schools, teach- 
ers, and students at the subgroup level (e.g., course-taking patterns of Hispanic male 12th 
graders); state-level results; reports for a set of large urban districts; history of state and 
district participation; and publicly released assessment questions, student responses, and 
scoring guides. The NAEP website (http://nces.ed.gov/nationsreportcard) contains user- 
friendly data analysis software to enable policymakers, researchers, and others to exam- 
ine all aspects of NAEP data, perform significance tests, and create customized graphic 
displays of NAEP results. These data and software tools can be used to inform policy- 
making and for secondary analyses and other research purposes. 

Challenges of Developing a NAEP Assessment 



There are three major challenges in developing a NAEP framework: (1) measurement 
constraints and the nature of the items included on the assessment, (2) time and resource 
constraints and how much can be assessed in NAEP, and (3) the timeline for the 
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framework and the difficulty of developing a 10-year framework with the rapid explosion 
of knowledge in the Information Age. Each of these challenges is discussed below. 

MEASUREMENT CONSTRAINTS 

Like any large-scale assessment in education, the workplace, or clinical practice, NAEP 
is constrained in what it can measure. This has implications for the proper interpretation 
of NAEP Science Assessment results. The NAEP Science Assessment framework is an 
assessment framework, not a curriculum framework. Although the two are clearly interre- 
lated, each has a different purpose and a different set of underlying assumptions. A cur- 
riculum framework is designed to inform instruction, to guide what is taught, and often, 
to guide how it is taught. It represents a very wide universe of learning outcomes from 
which teachers pick and choose what and how they teach. An assessment framework is a 
subset of the achievement universe from which assessment developers must choose to 
develop sets of items that can be assessed within time and resource constraints. Hence, 
the science content to be assessed by NAEP has been identified as that considered central 
to the Physical, Life, and Earth and Space Sciences. As a result, some important out- 
comes of science education that are difficult and time consuming to measure (such as 
habits of mind, sustained inquiry, and collaborative research), but valued by scientists, 
science educators, and the business community, will be only partially represented in the 
framework and in the NAEP Science Assessment. Moreover, the wide range of science 
standards in the guiding national documents that could be incorporated into the frame- 
work had to be reduced in number so as to allow some indepth probing of fundamental 
science content. As a result, the framework and the specifications represent a careful dis- 
tillation that is not a complete representation of the original universe of achievement out- 
comes desirable for science education. 

TIME AND RESOURCE CONSTRAINTS 

What NAEP can assess is limited by time and resources. Like most standardized assess- 
ments, NAEP is an “on-demand” assessment. It ascertains what students know and can 
do in a limited amount of time (50 minutes for paper-and-pencil questions and, for a sub- 
set of students sampled, an additional 30 minutes for hands-on performance or interactive 
computer tasks) and with limited access to resources (e.g., reference materials, feedback 
from peers and teachers, opportunities for reflection and revision). The national and state 
standards, however, contain goals that require extended time (days, weeks, or months). 
Therefore, to assess student achievement in the kinds of extended activities that are a cen- 
tral feature of the national and state standards and many science curricula, it would be 
necessary to know (for example) the quality of students’: 

• reasoning while framing their research questions; 

• planning for data collection and the execution of the plan; 
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• abilities to meet unpredictable challenges that arise during an actual, ongoing sci- 
entific investigation; 

• lines of argument in deciding how to alter their experimental approach in the light 
of new evidence; 

• engagement with fellow students and/or the teacher in interpreting an observation 
or result and deciding what to do about it; and 

• deliberations and reasoning when settling on the defensible conclusions that might 
be drawn from their work. 

Like other on-demand assessments, NAEP cannot be used to draw conclusions about stu- 
dent achievement with respect to the full range of goals of science education. States, dis- 
tricts, schools, and teachers can supplement NAEP and other standardized assessments to 
assess the full range of science education standards. In addition to describing the content 
and format of an examination, assessment frameworks like this one signal to the public 
and to teachers the elements of a subject that are important. The absence of extended 
inquiry in NAEP, however, is not intended to signal its relative importance in the curricu- 
lum. Indeed, because of the significance of inquiry in science education, the framework 
promotes as much consideration of inquiry as can be accomplished within the time and 
resources available for assessment. 

BALANCING CURRENT AND FUTURE CURRICULA 

The framework attempts to strike a balance between what can reasonably be predicted 
about future school science and what students are likely to encounter in their curriculum 
and instruction now and in the near future. It is a significant challenge to write a frame- 
work for the future. Cutting-edge science research creates new knowledge at the intersec- 
tion of disciplinary boundaries. For example, research on human and natural systems has 
generated new understanding about environmental science that is closely linked to 
knowledge generated in the Physical, Life, and Earth and Space Sciences. Although the 
framework is organized into the more traditional Physical, Life, and Earth and Space 
Sciences, features of current science research are woven throughout. 

Another example of burgeoning knowledge relates to technology and technological 
design and the role of both in the NAEP Science Assessment. Technology and techno- 
logical design are closely interrelated with science, yet the focus of this assessment is on 
science. Hence, technology and technological design are included in the framework but 
are limited to that which has a direct bearing on the assessment of students’ science 
achievement (see chapter three). 

The framework is intended to be both forward looking (in terms of the science content 
that will be of central importance in the future) and reflective (in terms of current school 
science). Because it is impossible to predict with certainty the shape of school science in 
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2009 and beyond, the choices made for 2009 should be revisited in response to future de- 
velopments in school science. 

Achievement Levels 



Public Law 107-279 specifies the Governing Board’s responsibilities regarding NAEP, 
including the identification of appropriate achievement goals for each age and grade in 
the subject areas assessed by NAEP. 

To carry out its mandated responsibility to set appropriate achievement goals for NAEP, 
the Governing Board adopted an achievement levels policy in 1989 (modified in 1993). 
This policy establishes three levels of achievement — Basic, Proficient, and Advanced. 
Basic denotes partial mastery of prerequisite knowledge and skills that are fundamental 
for proficient work at each grade. Proficient represents solid academic performance for 
each grade assessed. Students reaching this level have demonstrated competency over 
challenging subject matter, including subject-matter knowledge, application of such 
knowledge to real-world situations, and analytical skills appropriate to the subject matter. 
Advanced signifies superior performance. These levels are the primary means of report- 
ing NAEP results to the general public and policymakers regarding what students should 
know and be able to do on NAEP assessments. (See appendix B for the NAEP science 
preliminary achievement level descriptions and additional information about their devel- 
opment and use.) 

Introduction to the Assessment Framework 



Science comprises both content and practices. The NAEP Science Assessment provides a 
snapshot view of what the nation’s 4th, 8th, and 12th graders know and can do in science. 
One expects students, as a result of their education and life experiences, to have learned 
about the principles (along with the facts, concepts, laws, and theories) that have been 
verified by the community of scientists, as well as how scientists discover regularities in 
the natural world. 1 NAEP will assess students’ abilities to identify and use science princi- 
ples as well as use scientific inquiry and technological design (see chapter three). Al- 
though the framework distinguishes content from practice, the two are closely linked in 
assessment as in science itself. 

The framework addresses scientific knowledge and processes. Science is a way of know- 
ing about the natural world based on tested explanations supported by accumulated em- 
pirical evidence. Explanations of natural phenomena that rely on nonscientific views are 
not reflected in the framework. The committees responsible for the framework develop- 
ment relied on National Standards , Benchmarks, international frameworks, and state 
standards for content about the nature and practice of science. As stated in the National 
Standards: 



1 “Science principles” is used throughout this framework to denote not only the principles but also the facts, 
concepts, theories, and laws of science. 
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Scientific explanations must meet certain criteria. First and foremost, they 
must be consistent with experimental and observational evidence about 
nature, and must make accurate predictions, when appropriate, about sys- 
tems being studied. They should also be logical, respect the rules of evi- 
dence, be open to criticism, report methods and procedures, and make 
knowledge public (p. 201). 

The design of the NAEP Science Assessment is guided by the framework’s descriptions 
of the science content and practices to be assessed. Exhibit 1 illustrates how content and 
practices are combined (crossed) to generate performance expectations. The columns 
contain the science content (defined by content statements — propositions that express 
science principles — in three broad areas) and the rows contain the four science practices. 
A double dashed line distinguishes identifying science principles and using science prin- 
ciples from using scientific inquiry and using technological design. The first two prac- 
tices can be generally considered as “knowing science” and the last two practices can be 
considered as the application of that knowledge to “doing science” and “using science to 
solve real-world problems.” The cells at the intersection of content (columns) and prac- 
tices (rows) contain student performance expectations. Because content and practice 
categories are not entirely distinct, some overlap in the resultant performance expecta- 
tions is to be expected (as denoted by dashed lines). 
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Exhibit 1. Crossing content and practices to generate performance expectations 
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Exhibit 2 illustrates the fuller process of generating assessment items and interpreting 
student responses. An item is an individual question or exercise on the NAEP Science 
Assessment and is used to gather information about students’ knowledge and abilities. 
Exhibit 2 begins with student performance expectations, which describe in observable 
terms what students are expected to know and do on the assessment. These performance 
expectations guide the development of assessment items. The cognitive demands (see 
chapter three) of the items can then be used to interpret students’ responses as evidence 
of what students know and can do in science (see Ruiz-Primo et al. 2001 for more infor- 
mation on cognitive interpretations of performances on assessment tasks). Exhibit 2 sug- 
gests a linear process, but the development of an assessment is iterative (e.g., assessment 
items are modified based on student responses provided on trials of pilot versions). 
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Exhibit 2. Generating items and interpreting responses 




Overview of Framework Chapters 



This section provides an overview of the content in each chapter of the framework. For 
ease of understanding, three types of textboxes are used throughout the framework: clari- 
fication, crosscutting content, and illustrative item. Clarification textboxes provide details 
on potentially confusing topics, such as the distinction between mass and weight. Cross- 
cutting content textboxes provide descriptions of science content that cuts across the three 
broad content areas, such as biogeochemical cycles. Illustrative item textboxes provide 
assessment items that exemplify recommendations discussed in the text. 

CHAPTER TWO: SCIENCE CONTENT 

Key principles as well as facts, concepts, laws, and theories that describe regularities in 
the natural world are presented in chapter two as a series of content statements to be 
assessed at grades 4, 8, and 12. Taken together, these statements comprise the NAEP sci- 
ence content. They define only what is to be assessed by NAEP and are not intended to 
serve as a science curriculum framework. The content statements should be revisited pe- 
riodically in response to new developments in science research and school curriculum. 
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The content statements are organized according to the three broad content areas that gen- 
erally comprise the K-12 school curriculum: 

• Physical Science 

• Life Science 

• Earth and Space Sciences 

Classifying content statements into one content area is not always clearcut, but doing so 
helps ensure the areas of science are assessed in a balanced way. Some common content 
is found to be significant in each of the three content areas (e.g., energy conservation and 
its associated principles are applicable to the living and nonliving systems studied by 
physical, life, and Earth scientists). This crosscutting content is further described in 
chapter two. 

The content statements are derived from National Standards and Benchmarks and are in- 
formed by international frameworks and state standards. Content statements have been 
added where warranted by advances in science since the development of the standards 
documents. The selection of science content statements to be assessed at each grade level 
focuses on principles central to each discipline, tracks related ideas across grade levels, 
and limits the breadth of science knowledge to be assessed. The selection of content 
statements used an iterative approach and took into account the many perspectives of 
various stakeholders. 

CHAPTER THREE: SCIENCE PRACTICES 

The following science practices are found in most or all of the major sources used to de- 
velop the framework (see standards documents listed above). The practices to be assessed 
at grades 4, 8, and 12 are organized into four categories: 

• Identifying Science Principles 

• Using Science Principles 

• Using Scientific Inquiry 

• Using Technological Design 

As shown in exhibit 1, the Physical Science, Life Science, and Earth and Space Sciences 
content statements listed in chapter two can be deliberately combined (crossed) with each 
of the above four practices to generate specific performance expectations. These per- 
formance expectations are written in observable terms and guide the development of 
assessment items. Performance expectations thus provide an outline of what the NAEP 
Science Assessment will expect as evidence of what students know and can do. At the 
end of chapter three, examples of performance expectations, items, and interpretations of 
student responses are provided. (See the specifications for more examples.) 
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Students’ understanding increases over time as they learn more, moving from initially 
naive to increasingly more sophisticated knowledge about the natural world. These learn- 
ing progressions are further described in chapter three. 



CHAPTER FOUR: OVER VIEW OF THE ASSESSMENT DESIGN 

Chapter four provides an overview of the assessment design, including a discussion of 
assessment item contexts, types of items, distribution of items, accessibility concerns for 
English language learners and students with disabilities, and recommendations for small- 
scale special studies. 

Beyond the science content statements and practices, there are other valuable components 
of science that NAEP will not directly assess. These components — the history and nature 
of science and the relationship between science and technology — are treated as possible 
contexts in which assessment items may be presented. 

The assessment includes selected-response (multiple-choice) items and constructed- 
response items (which include short and extended constructed-response items as well as 
concept-mapping tasks). Some combination items may require more than one response. 
They include item clusters, predict-observe-explain (POE) item sets, hands-on perform- 
ance tasks, and interactive computer tasks. The responses requested may be all selected 
response, all constructed response, or a mixture of both. (See chapter four for a fuller ex- 
planation of types of items, including examples.) 

Chapter four recommends three types of percentages for item distribution (as measured 
by student response time) at each grade level: 

• Items by content area (Physical, Life, and Earth and Space Sciences) 

• Items by science practice (Identifying Science Principles, Using Science Princi- 
ples, Using Scientific Inquiry, and Using Technological Design) 

• Items by type (selected-response and constructed-response items) 

Additional specifications about the number of hands-on performance tasks and interactive 
computer tasks are also provided. 

Chapter four also describes considerations for English language learners and students 
with disabilities. In particular, NAEP assessments need to be responsive to growing de- 
mands for increased inclusion of all types of students in the general curriculum and in- 
creased emphasis and commitment to serve and be accountable for all students. A number 
of small-scale special studies also are recommended at the end of chapter four. 
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Comparing the NAEP Science Frameworks: 1996-2005 2009 



The chapter overviews provided above reflect the differences between the NAEP 1996— 
2005 science frameworks and this new framework to be used for 2009 and future NAEP 
science assessments. The assessment resulting from this 2009 framework will start a new 
NAEP science trend (i.e., measure of student progress in science). One difference be- 
tween the two frameworks is that the 2009 steering and planning committees drew on a 
variety of new standards and assessments. They also were able to extract findings from 
research in science, science education, and cognition and to consider the use of technol- 
ogy to increase the options for assessment administration. Exhibit 3 lists the major differ- 
ences between the 1996-2005 and 2009 NAEP science frameworks. 

Exhibit 3. NAEP science frameworks: 1996-2005 -> 2009 





1996-2005 Frameworks 


2009 Framework 




Few science standards available on which to 
base the content to be assessed 


Content drawn from existing standards and 
assessment frameworks: National 
Standards, Benchmarks , TIMSS, PISA, and 
state standards 




Content areas organized into Physical 
Science, Life Science, and Earth Science 


Content areas organized into Physical 
Science, Life Science, and Earth and Space 
Sciences 


S3 

S3 

O 

U 

c J 


Recommendations on distribution of 
questions by fields of science and grade: 
approximately equal distribution in grades 4 
and 12; a somewhat heavier emphasis on 
Life Science in grade 8 


Recommendations on distribution of 
questions by science content area and grade: 
equal weight for all three sciences in grade 
4; emphasis on Earth and Space Sciences in 
grade 8; emphasis on Physical Science and 
Life Science at grade 12 


S3 

.a 

m 


Content presented as bullets and short 
phrases 


Content presented as statements in tables 
organized by science content subtopics (e.g., 
forces affecting motion from Physical 
Science) and by grade level 




Framework employed three abstract themes: 
systems, models, and patterns of change 








Framework employs crosscutting content 
among Life, Physical, and Earth and Space 
Sciences 




Assessment asked questions about the nature 
of science 
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Exhibit 3. NAEP science frameworks: 1996-2005 -> 2009 (cont.) 





1996-2005 Frameworks 


2009 Framework 




Knowing and doing dimension organized 
into conceptual understanding, scientific 
investigation, and practical reasoning 


Science practices dimension organized into 
Identifying Science Principles, Using 
Science Principles, Using Scientific Inquiry, 
and Using Technological Design 






Nature of science treated within science 
practices, particularly Using Science 
Principles and Using Scientific Inquiry 


c /> 

cj 


Science practices assessed were largely 
experience based 


Science practices assessed take into account 
extant research and the cognitive complexity 
of the items 


u 

a 

u 

Ph 

a> 

u 


Assessment included items on practical 
reasoning (i.e., applying science to suggest 
effective solutions to everyday problems) 




S3 

Q) 

’D 

n 




Assessment includes questions on 
technological design (i.e., the systematic 
process of applying science knowledge and 
skills to solve problems in a real-world 
context) 




Forty-five percent of the assessment focused 
on conceptual understanding 


Sixty percent of the assessment focuses on 
conceptual understanding (Identifying and 
Using Science Principles) 






Learning progressions (i.e., connected 
sequences of science performances across 
grade spans) are included 






Assessment uses the history of science and 
the relationship between science and 
technology as contexts for questions 


C/3 


Assessment included both paper-and-pencil 
and hands-on performance tasks 


Assessment includes paper-and-pencil 
questions, hands-on performance tasks, and 
interactive computer tasks 


E 

QJ 

S3 

QJ 

E 

C/3 
C A 


No illustrative items to convey science 
knowledge or practices in the framework; 
only a few suggested ideas for items 
provided in the specifications 


Illustrative items that convey science 
knowledge and practices are included in 
both the framework and the specifications 


QJ 

CZ3 

C/3 

◄ 




Framework and specifications include 
guidelines for assessing students with 
disabilities and English language learners 






Framework includes examples showing how 
questions are generated and interpreted 






Students’ naive conceptions about science 
principles arc explicitly assessed 
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Chapter Two 



Science Content 

This chapter presents a series of statements that describe the science content of the 2009 
NAEP Science Assessment. The content statements contain key science principles for 
NAEP assessment. In the framework, the phrase “science principles” is broadly con- 
ceived and encompasses not only the key principles but also the facts, concepts, laws, and 
theories of science. To specify the science that should be assessed at each grade level, the 
framework organizes the science content into the three broad content areas that generally 
make up the K-12 school curriculum to which students are exposed: 

• Physical Science 

• Life Science 

• Earth and Space Sciences 

Classifying statements into one primary content area is not always clearcut and is artifi- 
cial to some extent. For example, physicist Ernest Rutherford’s discovery of the nucleus 
earned him the Nobel Prize in Chemistry and physicist Rosalyn Yalow’s work on radio- 
immunoassay earned her the Nobel Prize in Medicine. However, using three broad con- 
tent areas as an organizer helps ensure that key science content is assessed in a balanced 
way. 

For clarity, exhibits are used to depict the content statements at each grade level. The 
content statements are based on the assumption that a person literate in science is one 
who understands key science ideas, is aware that science and technology are interdepend- 
ent human enterprises with strengths and limitations, is familiar with the natural world 
and recognizes both its diversity and unity, and uses scientific knowledge and ways of 
thinking for individual and social purposes (see AAAS 1994, p. xvii). 

Two types of textboxes are used throughout this chapter. Clarification textboxes provide 
details on potentially confusing content, such as the distinction between mass and weight. 
Crosscutting content textboxes provide descriptions of science content that cuts across 
the three broad content areas, such as biogeochemical cycles. 



Development of the Content Statements 



The selection and generation of specific content statements at each grade level followed a 
similar approach across the three broad content areas: 

• The National Standards and Benchmarks were used as key documents for 
identifying the science content to be assessed, pursuant to the charge from the 
steering committee. Various tools, primarily crosswalks between the National 
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Standards and Benchmarks (AAAS 1997; Kendall and Marzano 2004), were used 
to cross-check the documents’ content standards and benchmark statements; those 
that were common to both documents were generally given priority. On a case- 
by-case basis, content not represented in both documents was discussed and 
decisions made about inclusion or exclusion. Additions were made where 
warranted by scientific advances in the decade or more since the development of 
the National Standards and Benchmarks, or as a consequence of international 
assessment results from TIMSS and PISA. Some of this framework’s content 
statements are verbatim reproductions of statements from the National Standards 
and Benchmarks } 

• The focus in the selection process was on the central principles of each discipline. 
The content statements in the framework represent foundational and pervasive 
knowledge, key points of scientific theories, and underpinnings upon which com- 
plex understanding is built; and/or they demonstrate connectivity to other central 
content. 

• A primary consideration was the grade-level appropriateness and accuracy within 
grade level of content statements. 

• Once key content was identified within subtopics, the progression of ideas and 
performances, informed by available research, was tracked through grades 4, 8, 
and 12. 

• A deliberate attempt was made to limit the breadth of science content to be as- 
sessed so that some important topics could be measured indepth. Once core con- 
tent was identified in each science area, additional content statements could be 
added only if others previously included were eliminated. 

The selection and generation of content statements for inclusion in the framework was 
not a linear process. Although the planning committee attempted to use clear and concise 
language, the complexities associated with the task of defining what students should 
know and be able to do in science at particular points in their development necessitated 
an iterative approach that included many perspectives. In addition to internal review by 
the project committees and staff, outreach activities gathered external feedback on the 
content statements from a variety of stakeholders (e.g., teachers, school and district ad- 
ministrators, state science education personnel, policymakers, scientists, and members of 
the business, industry, and postsecondary communities). The framework should be revis- 
ited in the future as new research becomes available and as the influence of new devel- 
opments in science takes shape in the K-12 curriculum. 



2 Statements are reprinted from National Science Education Standards (NRC 1996) with permission from 
the National Academy of Sciences, courtesy of the National Academies Press, Washington, DC. Statements 
are reprinted from Benchmarks for Science Literacy (AAAS 1993) with permission from Project 2061, on 
behalf of the American Association for the Advancement of Science, Washington, DC. 
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Organization of Science Content 



As described above, this framework organizes science content into three broad content 
areas (Physical Science, Life Science, and Earth and Space Sciences). The content is fur- 
ther organized into topics (such as motion), subtopics (such as forces affecting motion), 
and grade- specific content statements. The description of each broad content area follows 
this structure of increasing specificity and is presented in two ways: narrative introduc- 
tions and content statements presented in exhibits (see exhibits 8, 10, and 12 in the sec- 
tions for Physical Science, Life Science, and Earth and Space Sciences, respectively). 
Exhibit 4 summarizes the 2009 NAEP science content topics and subtopics. 

Exhibit 4. 2009 NAEP science content topics and subtopics 



Physical 

Science 


Life 

Science 


Earth and Space 
Sciences 


Matter 


Structures and Functions 


Earth in Space and Time 


• Properties of matter 


of Living Systems 


• Objects in the 


• Changes in matter 


• Organization and 


universe 




development 


• History of Earth 


Energy 


• Matter and energy 




• Forms of energy 


transformations 


Earth Structures 


• Energy transfer and 


• Interdependence 


• Properties of Earth 


conservation 




materials 




Changes in Living 


• Tectonics 


Motion 


Systems 




• Motion at the 


• Heredity and 


Earth Systems 


macroscopic level 


reproduction 


• Energy in Earth 


• Forces affecting 


• Evolution and 


systems 


motion 


diversity 


• Climate and weather 

• Biogeochemical 
cycles 



As an organizational tool in exhibits 8, 10, and 12, each content statement is preceded by 
a specific code in bold (e.g., “L12.10: Sorting and recombination of genes in sexual re- 
production results in a great variety of possible gene combinations from the offspring of 
any two parents”). Within a code, the letter denotes broad content area (“P” for Physical 
Science, “L” for Life Science, and “E” for Earth and Space Sciences); the number before 
the period denotes grade level (grade 4, 8, or 12); and the number following the period 
denotes the content statement’s order of appearance within a given content area and 
grade. Thus, L12.10 denotes that this is the tenth content statement to appear in the grade 
12 section of the Life Science content statements table. Because the numbering within 
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each content area and grade is strictly sequential, code numbers do not necessarily indi- 
cate any relationships across grades (see, for example, P4.13, P8.13, and P12.13). 



Interpretation of the Content Statements 



In the framework, the content statements generally follow a form that is consistent with 
the National Standards , Benchmarks, and the practice of the scientific community. The 
content statements are phrased as propositions that express science principles. Based on 
evidence, these principles have been verified by the scientific community and are under 
constant review. Exhibit 5 is an example of how one grade 8 Physical Science principle is 
represented in the framework, National Standards, and Benchmarks. 



Exhibit 5. One grade 8 Physical Science principle represented in the framework, 

National Standards, and Benchmarks 



Framework 


National Standards 


Benchmarks 


P8.16: Forces have magnitude 
and direction. Forces can be 
added. The net force on an object 
is the sum of all the forces acting 
on the object. A nonzero net force 
on an object changes the object’s 
motion; that is, the object’s speed 
and/or direction of motion 
changes. A net force of zero on 
an object does not change the 
object’s motion; that is, the object 
remains at rest or continues to 
move at a constant speed in a 
straight line (p. 35).* 


If more than one force acts on an 
object along a straight line, then 
the forces will reinforce or cancel 
one another, depending on their 
directions and magnitude. 
Unbalanced forces will cause 
changes in the speed or direction 
of an object’s motion (p. 154). 


An unbalanced force acting on an 
object changes its speed or 
direction of motion, or both. If 
the force acts toward a single 
center, the object’s path may 
curve into an orbit around the 
center (p. 90). 



* This content statement is longer in the framework than in the Benchmarks or National Standards — not 
because it introduces additional science principles, but because it has adopted more detailed language. 



The content statements form the basis for explaining or predicting naturally occurring 
phenomena. For example, the above content statement about objects in motion can be 
used to explain and predict the motions of many different specific objects (e.g., an ice 
skater, an automobile, an electron, or a planet). 

The content statements do not include observations of phenomena. As the content state- 
ments are written, the empirical foundations of the science principles they represent are 
not detailed. Instead, knowledge is presented in general terms, such as patterns in obser- 
vations or theoretical models that account for these patterns. The NAEP assessment will 
require students to apply content statements to specific observations and examples of 
phenomena. Therefore, the specifications provide boundaries on the range of phenomena 
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and examples that are appropriate for inclusion on the NAEP Science Assessment. The 
boundaries also describe the appropriate instrumentation, representations, and technical 
vocabulary to be included in the assessment. 

The process of item development and making inferences about students’ knowledge and 
abilities (see exhibits 1 and 2) may necessitate further clarification of the content state- 
ments themselves. For example, this may involve “detailing” the meanings of individual 
content statements (e.g., “boiling point” assumes standard atmospheric pressure) or fur- 
ther defining the boundaries of the content to be assessed (e.g., grade 12 students are ex- 
pected to know that DNA provides instructions for assembling proteins but not the details 
of DNA transcription and translation). Exhibit 6 is an example of commentary on a grade 
8 Physical Science content statement. Although the commentary describes key ideas 
about waves relevant through grade 12, only three of these key ideas are recommended 
for assessment at grade 8. Whenever possible, commentary on content statements was 
informed by available research on learning and cognition. Commentary is intended as 
helpful notes to item writers and is provided for a number of content statements in the 
specifications. Commentary is not always of the same kind, and the examples provided 
do not exhaustively cover the content statements. Assessment developers should continue 
the process of “detailing” the grade- appropriate principles to be assessed for all content 
statements sampled for a particular NAEP Science Assessment. 

Exhibit 6. Commentary on a Physical Science content statement 

Grade 8: Energy — Forms of Energy 

Content Statement 

P8.10: Energy is transferred from place to place. Light energy from the Sun travels 
through space to Earth (radiation). Thermal energy travels from a flame through the metal 
of a cooking pan to the water in the pan (conduction). Air warmed by a fireplace moves 
around a room (convection). Waves — including sound and seismic waves, waves on 
water, and light waves — have energy and transfer energy when they interact with matter. 

Commentary 

Wave principles recommended for assessment at grade 8: 

• Waves involve transfer of energy without a transfer of matter. 

• Waves are caused by disturbances and are also themselves disturbances. Some of 
the energy of these disturbances is transmitted by the wave. 

• Water, sound, and seismic waves transfer energy through a material. 
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Exhibit 6. Commentary on a Physical Science content statement (cont.) 



Wave principles that are related but not recommended for assessment at grade 8: 

• Some waves are transverse (water, seismic) and other waves are longitudinal 
(sound, seismic). 

• In transverse waves, the direction of the motion is perpendicular to the 
disturbance. 

• In longitudinal waves, the direction of motion is parallel to the disturbance. 

• Waves (e.g., light waves) traveling from one material to another undergo 
transmission, reflection, and/or changes in speed. 

• Waves can be described by their wavelength, amplitude, frequency, and speed 
(speed is frequency multiplied by wavelength; energy is a function of the 
amplitude for nonelectromagnetic waves). 

• Light has dual wave-particle properties. 

Energy and refraction calculations are not recommended for assessment at grade 8. A 
quantitative understanding of electromagnetic waves is expected at grade 12. See P12.10. 



To fully understand the content statements and their intent, readers of the framework 
should be cognizant of the following: 

• Although all content statements have been assigned a primary classification, some 
are likely to fall into more than one content area. 

• Some assessment items may draw on more than one content statement at a time. 

• Empty cells in the content statement tables denote that a particular subtopic is not 
recommended for assessment at that grade level. 

• Retention of foundational knowledge from one grade to the next is assumed; 
however, if the relevant content statement does not appear in a succeeding grade 
level, it should not be assessed. 

• The content statements listed in the framework describe the whole of what is to be 
assessed on the 2009 NAEP. The content statements should not be interpreted as a 
complete description of the recommended school science curriculum. 

Crosscutting Content 



Scientists define their specializations narrowly (e.g., astronomy, molecular biology, 
organic chemistry) to organize their research communities; the categories of Physical 
Science, Life Science, and Earth and Space Sciences are helpful for organizing school 
science. These categorizations mask the fact that some science principles cut across the 
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content areas. In this framework, crosscutting content is not represented by abstractions 
such as “models,” “constancy and change,” or “form and function,” but is anchored in the 
content statements themselves. Some examples of crosscutting content are described in 
textboxes that appear throughout the content area introductions: Energy Sources and 
Transfer in Physical Science; Uses, Transformations, and Conservation of Energy in Life 
Science; and Biogeochemical Cycles in Earth and Space Sciences. Additional examples 
are suggested in the specifications. For instance, the theory of plate tectonics and the 
evolution of Earth’s surface are inextricably linked with environmental pressures (such as 
geographic barriers), speciation, and the evolution of life. Such examples illustrate 
opportunities for assessing specific content in greater depth. 

Physical Science 



Physical Science principles, including fundamental ideas about matter, energy, and 
motion, are powerful conceptual tools for making sense of phenomena in physical, living, 
Earth, and space systems. Familiar changes — an ice cube melting, a baseball changing 
direction after being struck by a bat, the appearance of a bolt of lightning, the formation 
and erosion of mountains, and the growth of a plant — can be explained using these 
fundamental ideas. 3 

Energy is the constant in an ever-changing world. Energy from the Sun fuels electrical 
storms, hurricanes, tornados, and photosynthesis. In turn, the products of photosynthesis 
(carbohydrates and oxygen) react during respiration to fuel life processes, such as growth 
and reproduction of plants and animals. Consequently, it is important for students to de- 
velop an understanding of Physical Science principles early and to appreciate their use- 
fulness across Physical Science, Life Science, and Earth and Space Sciences. 

The Physical Science principles to be assessed are divided into three topics — matter, 
energy, and motion. Matter is the “stuff’ of the natural world. Energy is involved in all 
changes in matter. Motion of the heavenly bodies, of objects found in daily experiences 
(e.g., balls, birds, cars), and of the tiny particles (atoms, molecules, and their component 
parts) composing all objects and substances is the result of interactions of matter and en- 
ergy. The content statements have been divided into topics and subtopics as summarized 
in exhibit 7. 



3 The importance of developing an understanding early and making connections among the Physical, Life, 
and Earth and Space Sciences has led to increased attention on Physical Science in elementary school and 
prompted consideration of rearranging the usual Earth-life-chemistry-physics curriculum sequence. 
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Exhibit 7. Physical Science content topics and subtopics 



Matter 



Properties of matter 
Changes in matter 

Energy 

Forms of energy 

Energy transfer and conservation 

Motion 



Motion at the macroscopic level 
Forces affecting motion 



MATTER 

The topic “matter” is divided into two subtopics: properties of matter and changes in mat- 
ter. Conservation of mass, the particulate model of matter, and the periodic table of the 
elements are the concepts that connect these two subtopics and their related principles. 

Properties of Matter 

Matter has physical and chemical properties. Physical properties common to all matter as 
well as those physical properties unique to solids, liquids, and gases are included in the 
framework, as are chemical properties. All objects and substances in the natural world are 
composed of matter. Matter has two fundamental properties: it takes up space and it has 
inertia; that is, it changes motion only when under the influence of a nonzero net force 
(grade 4). See the following textbox, “Clarification: A Matter of Mass,” for more on mass 
versus weight and the framework’s treatment of this distinction. 
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Clarification: A Matter of Mass 

Mass is a property common to all objects. It is the amount of matter (or “stuff’) in an 
object. The more mass an object has, the more inertia (or sluggishness) it displays when 
attempts are made to change its speed or direction. Mass is measured in grams (g) or 
kilograms (kg) (1 kg = 1,000 g) using a beam or electronic balance.* 

Weight, on the other hand, is a measure of the force of attraction (gravitational force) 
between an object and Earth. Every object exerts gravitational force on every other 
object. The force depends on how much mass the objects have and on how far apart they 
are. Force and weight are measured in newtons (N) using a spring scale. 

Changing an object’s position (e.g., from Earth to the Moon) will change its weight, but 
not its mass. For example, on the surface of Earth, a cannon ball has a mass of 10 kg and 
a weight of 98 N. On the surface of the Moon, the cannon ball still has a mass of 10 kg, 
but its weight is only 16 N. So, the cannon ball weighs less on the Moon than on Earth, 
even though nothing has been taken away. Why? Because of the Moon’s lesser mass and 
smaller radius, the force of attraction between the Moon and the cannon ball is less than 
the force of attraction between Earth and the cannon ball. Hence, it is said that an object 
on the Moon weighs less than the same object weighs on Earth. 

These concepts of mass and weight are complicated and potentially confusing to fourth 
grade students. Therefore, this framework uses the more familiar term “weight” in grade 
4 to stand for both weight and mass, and this usage is denoted as “weight (mass).” By 
grades 8 and 12, students are expected to understand the distinction between mass and 
weight, and both terms will be used as appropriate. 

* In current NAEP practice, metric units of measure are used for grades 4, 8, and 12. 



Matter exists in several different physical states, each of which has unique properties 
(grade 4). Three of the most commonly encountered states are solids, liquids, and gases. 
Shape and compressibility are examples of properties that distinguish solids, liquids, and 
gases (grade 4). 

The particulate model of matter can be used to explain and predict the properties of states 
of matter, such as why ice is harder than liquid water and why ice (once formed) has a 
shape independent of its container while liquid water takes the shape of whatever 
container it is in (grade 4). In the particulate model of matter, the molecules or atoms of 
which matter is composed are assumed to be tiny particles in motion (grade 8). The 
motion is translational, rotational, and vibrational (grade 12). This model can be used to 
explain the properties of solids, liquids, and gases as well as changes of state. The 
particulate model can be used to explain the unique properties of water, as described in 
the following textbox. 
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Clarification: Unique Properties of Water 

Grade 12: Matter — Changes in Matter 

P12.5: Changes of state require a transfer of energy. Water has a very high specific heat, 
meaning it can absorb a large amount of energy while producing only small changes in 
temperature. 

The unique properties of water have important consequences for Earth systems and Life 
Science, including the origin and existence of life on Earth. Understanding the substance 
of water requires knowledge across the Physical Science categories of Matter, Energy, 
and Motion. 

As with all kinds of matter, water’s unique properties can be explained by the shape of its 
molecule, the forces between its molecules in solid (ice) and liquid states, and the 
resulting arrangement of molecules in solid and liquid states. In particular, in the solid 
state, the molecular bonds are such that they separate the molecules more than in the 
liquid state, resulting in ice being less dense than liquid water. The strong intermolecular 
forces of liquid water account for its high specific heat. (The specific heat of a substance 
is the amount of energy required to change 1 g of the substance by 1 degree Centigrade.) 



The detailed structures of molecules and the atoms that compose them serve as models 
that explain the forces of attraction between molecules. The structure of atoms, especially 
the outermost electrons, explains the chemical properties of the elements and the forma- 
tion of the chemical bonds made and broken during chemical reactions (grade 12). The 
Periodic Table of the Elements (introduced at grade 8) is another way in which order can 
be made out of the complexity of the variety of types of matter. (In grade 8, the emphasis 
is on observed periodicity of properties.) The Periodic Table demonstrates the relation- 
ship between the atomic number of the elements and their chemical and physical proper- 
ties and provides a structure for inquiry into the characteristics of the chemical elements 
(grade 12). 

Two classes of chemical substances serve as exemplars of chemical properties: metals 
(elements) and acids (compounds). A chemical property of metals is to react with non- 
metals to form salts. One of the properties of acids is the formation of characteristic col- 
ors when interacting with acid/base indicators and the interaction with bases to produce 
salts and water (i.e., neutralization) (grade 8). 

Changes in Matter 

Matter can undergo a variety of changes. Changes are physical if the relationships 
between the molecules of the material are changed, such as from a solid to a liquid or 
from a liquid to a gas (grade 4). When matter undergoes a physical change, generally no 
changes occur in the structure of the molecules or atoms composing the matter (grade 8), 
although there are exceptions (e.g., sulfur). Changes are chemical if they involve the 
rearrangement of how atoms are bound to one another, thereby changing the molecules of 
the material. These are changes in the configuration of the outermost electron shell 
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surrounding the nuclei of the interacting atoms (grade 12). Changes are nuclear if the 
particles are emitted from or absorbed into the nucleus of the atom, thereby changing the 
atoms themselves into isotopes or different elements (grade 12). 

The fact that mass is conserved when matter undergoes physical and chemical changes is 
a powerful principle for understanding the natural world and was influential in the devel- 
opment of chemical theory. Adherence to the principle discourages the conclusion that 
something “disappears” (as water seems to disappear from a puddle) and encourages the 
search for the “missing” matter. 

Most nuclear reactions, involving changes in the nuclei of atoms, are very high energy 
and result in the formation of elements or nuclei different from those that began the 
process. In nuclear reactions, a measurable amount of mass is converted into energy 
(grade 12). 

ENERGY 

The topic “energy” is divided into two subtopics; one addresses the forms of energy and 
the other addresses energy transfer and conservation. 4 

Forms of Energy 

Knowing the characteristics of familiar forms of energy (grade 4) and the scientific cate- 
gories of potential and kinetic energy (grade 8) can lead to an understanding that, for the 
most part, the natural world can be explained and is predictable. The most basic charac- 
teristics of thermal, light, sound, electrical, and mechanical energy and the relationship 
between changes in the natural world and energy are included in the framework. 5 For ex- 
ample, the fact that two objects, one at a higher temperature than the other, come to the 
same temperature when placed in contact with each other is a familiar experience. Heat 
as a concept can be used to explain this experience (grade 8). 



Energy Transfer and Conservation 

The fact that energy is conserved can be demonstrated by keeping track of the familiar 
forms of energy as they are transferred from one object to another. The chemical poten- 
tial energy in a battery is transferred by electric current to a light bulb, which in turn 
transfers the energy in the form of heat (thermal energy) and light to its surroundings 
(grade 4). The energy stored in the battery decreases as its surroundings are heated. The 
loss in chemical potential energy equals the light and heat (thermal energy) transferred by 



4 There are different approaches to helping students understand the concept of energy and related observ- 
able phenomena, such as light, heat, and sound. These differences are reflected in the framework's refer- 
ence documents and have therefore influenced the framing of the Physical Science, Life Science, and Earth 
and Space Sciences content statements. 

5 The term “heat” in Physical Science is used to stand for thermal energy in grade 4; this usage is denoted 
as “heat (thermal energy).” “Thermal energy” is used in grades 8 and 12. 
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the bulb and the wires to their surroundings. Quantitative accounting is complex; how- 
ever, on a qualitative basis, both the ability to trace energy transfer and the understanding 
that energy is conserved (grade 8) are of great explanatory and predictive value. Chemi- 
cal reactions either release energy to the surroundings or cause energy to flow from the 
surroundings into the system (grade 12). The Sun as the main energy source for the Earth 
provides an opportunity at all grade levels to make important connections between the 
science disciplines (see the following textbox). 



NAEP 2009 Science Framework 



30 




Crosscutting Content: Energy Sources and Transfer 

When a sufficiently high temperature occurs in the Sun because of gravitational 
attraction, nuclear reactions take place. These reactions result in the transfer of energy 
from nuclei to their surroundings. At the same time, those high temperatures cause the 
Sun to radiate visible light and many other forms of electromagnetic waves. A small 
fraction of this light energy reaches Earth, heating the land, air, and water. Some of this 
energy causes some water to evaporate. The water vapor travels high into the atmosphere, 
thus increasing its gravitational potential energy. There it cools and condenses, some of it 
falling into reservoirs behind dams. At many dams, some of this water is directed 
downhill through tubes, resulting in the transfer of gravitational potential energy to the 
descending water as kinetic energy. This water is then used to turn turbines, and energy is 
transferred from the moving water to electrical appliances through circuits and power 
lines. Accordingly, the energy used to power commonplace objects such as a light bulb, 
TV, radio, or stereo can be traced back to nuclear reactions deep inside the Sun. 

Consider also the transfer of energy that occurs as a diver falls through air into water. 
When the diver is initially poised on a cliff high above a lake’s surface, one says that the 
diver has potential energy with respect to the air and water below. As the diver falls, her 
speed (kinetic energy) increases as her potential energy decreases. Her body transfers 
energy to the medium through which she falls; that is, the diver’s body rubs against the 
air and water (heating them) and exerts force on the air and water (moving them aside). 
When the diver finally comes to rest in the lake, some or most of her potential energy has 
been transferred to heating and setting into motion the air and water through which she 
fell. 



The following grade 12 content statements illustrate the crosscutting nature of energy 
sources and transfer. They are not intended to represent an exhaustive catalog of all 
statements related to this crosscutting content. 



Physical Science 


Earth and Space Sciences 


P12.ll: Fission and fusion are reactions involving 
changes in the nuclei of atoms. Fission is the 
splitting of a large nucleus into smaller nuclei and 
particles. Fusion involves joining of two relatively 
light nuclei at extremely high temperature and 
press lire. Fusion is the process responsible for the 
energy of the Sun and other stars. (Underlining is 
used here and on pp. 42 and 56 to link segments of 


E12.9: Earth systems have internal and external 
sources of energy, both of which create heat. The 
Sun is the major external source of energy. Two 


primary sources of internal energy are the decay of 
radioactive isotopes and the gravitational energy 
from Earth's original formation. 


the content statements across content areas.) 
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MOTION 



The topic “motion” is divided into two subtopics; one addresses motion at the macro- 
scopic level and the other addresses the forces that affect motion. 

Motion at the Macroscopic Level 

Objects observed in daily life undergo different kinds of motion (grade 4). The frame- 
work distinguishes three kinds of motion (translational, rotational, and vibrational) and 
emphasizes the translational motion of objects in the natural environment (grade 12). 
Translational motion is more difficult to describe than it appears because descriptions de- 
pend on the position of the observer and the frame of reference used. Speed (grades 4 and 
8), velocity (grade 12), and acceleration (grade 12) of objects in translational motion are 
described in terms of change in direction and position in a time interval. 

Forces Affecting Motion 

It takes energy to change the motion of objects. The energy change is understood in terms 
of forces. For example, it takes energy for a baseball pitcher to set the ball in motion to- 
ward the batter. Also, pushes and pulls applied to objects often result in changes in mo- 
tion (grade 4). Principles germane to the relationship of forces and motion serve to moti- 
vate the search for forces when objects change their motion or when an object remains at 
rest even though it seems that the forces acting on it should result in setting it in motion 
(grade 8). 

Some forces act through physical contact of objects while others act at a distance. The 
force of a bat on a ball and the downward push of a lead block resting on a tabletop are 
contact forces. Gravitational and magnetic forces act at a distance (grade 8). Magnets do 
not need to be in contact to attract or repel each other. The Earth and an airplane do not 
need to be in contact for a force of attraction to exist between them. Qualitative relation- 
ships (grade 8) and quantitative relationships (grade 12) between the mass of an object, 
the magnitude and direction of the net force on the object, and its acceleration are power- 
ful ideas to explain and predict changes in the natural world. 
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Exhibit 8. Physical Science content statements for grades 4, 8, and 12 



Grade 4 


Grade 8 


Grade 12 


Matter 


Properties of Matter: Physical prof 
common to solids, liquids, and gases 
Table of the Elements (8); characteri 


terties common to all objects and subs 
; (4); chemical properties, particulate i 
sties of subatomic particles and atomi 


dances and physical properties 
nature of matter, and the Periodic 
lc structure (12). 



P4.1: Objects and substances have 
properties. Weight (mass) and volume 
are properties that can be measured 
using appropriate tools.* 

P4.2: Objects vary in the extent to 
which they absorb and reflect light 
and conduct heat (thermal energy) and 
electricity. 

P4.3: Matter exists in several different 
states; the most common states are 
solid, liquid, and gas. Each state of 
matter has unique properties. For 
instance, gases are easily compressed 
while solids and liquids are not. The 
shape of a solid is independent of its 
container; liquids and gases take the 
shape of their containers. 

P4.4: Some objects are composed of a 
single substance; others are composed 
of more than one substance. 

P4.5: Magnets can repel or attract 
other magnets. They can also attract 
certain nonmagnetic objects at a 
distance. 



P8.1: Properties of solids, liquids, and 
gases are explained by a model of 
matter that is composed of tiny 
particles in motion. 

P8.2: Chemical properties of 
substances are explained by the 
arrangement of atoms and molecules. 

P8.3: All substances are composed of 
1 or more of approximately 100 
elements. The periodic table organizes 
the elements into families of elements 
with similar properties. 

P8.4: Elements are a class of 
substances composed of a single kind 
of atom. Compounds are composed of 
two or more different elements. Each 
element and compound has physical 
and chemical properties, such as 
boiling point, density, color, and 
conductivity, which are independent 
of the amount of the sample. 

P8.5: Substances are classified 
according to their physical and 
chemical properties. Metals and acids 
are examples of such classes. Metals 
are a class of elements that exhibit 
common physical properties such as 
conductivity and common chemical 
properties such as reacting with 
nonmetals to produce salts. Acids are 
a class of compounds that exhibit 
common chemical properties, 
including a sour taste, characteristic 
color changes with litmus and other 
acid/base indicators, and the tendency 
to react with bases to produce a salt 
and water. 



P12.1: Differences in the physical 
properties of solids, liquids, and gases 
are explained by the ways in which 
the atoms, ions, or molecules of the 
substances are arranged and the 
strength of the forces of attraction 
between the atoms, ions, or 
molecules. 

P12.2: Electrons, protons, and 
neutrons are parts of the atom and 
have measurable properties, including 
mass and, in the case of protons and 
electrons, charge. The nuclei of atoms 
are composed of protons and 
neutrons. A kind of force that is only 
evident at nuclear distances holds the 
particles of the nucleus together 
against the electrical repulsion 
between the protons. 

P12.3; In the Periodic Table, elements 
are arranged according to the number 
of protons (called the atomic number). 
This organization illustrates 
commonality and patterns of physical 
and chemical properties among the 
elements. 

P12.4: In a neutral atom, the 
positively charged nucleus is 
surrounded by the same number of 
negatively charged electrons. Atoms 
of an element whose nuclei have 
different numbers of neutrons are 
called isotopes. 



* See textbox on p. 27 for more information on the distinction between weight and mass. 

* Although this content statement generally holds true, some compounds decompose before boiling. 
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Exhibit 8. Physical Science content statements for grades 4, 8, and 12 (cont.) 



Grade 4 


Grade 8 


Grade 12 


Matter 


Changes in Matter: Changes of stai 
particulate nature of matter, unique \ 
level during chemical changes (12). 


te (4); physical and chemical changes 
physical characteristics of water, and c 


and conservation of mass (8); 
:hanges at the atomic and molecular 



P4.6: One way to change matter from 
one state to another and back again is 
by heating and cooling. 



P8.6: Changes of state are explained 
by a model of matter composed of 
tiny particles that are in motion. When 
substances undergo changes of state, 
neither atoms nor molecules 
themselves are changed in structure. 
Mass is conserved when substances 
undergo changes of state. 

P8.7: Chemical changes can occur 
when two substances, elements, or 
compounds react and produce one or 
more different substances whose 
physical and chemical properties are 
different from the reacting substances. 
When substances undergo chemical 
change, the number and kinds of 
atoms in the reactants are the same as 
the number and kinds of atoms in the 
products. Mass is conserved when 
substances undergo chemical change. 
The mass of the reactants is the same 
as the mass of the products. 



P12.5: Changes of state require a 
transfer of energy. Water has a very 
high specific heat, meaning it can 
absorb a large amount of energy while 
producing only small changes in 
temperature. * 

P12.6: An atom’s electron 
configuration, particularly of the 
outermost electrons, determines how 
the atom can interact with other 
atoms. The interactions between 
atoms that hold them together in 
molecules or between oppositely 
charged ions are called chemical 
bonds. 

P12.7: A large number of important 
reactions involve the transfer of either 
electrons (oxidation/reduction 
reactions) or hydrogen ions (acid/base 
reactions) between reacting ions, 
molecules, or atoms. In other 
chemical reactions, atoms interact 
with one another by sharing electrons 
to create a bond. An important 
example is carbon atoms, which can 
bond to one another in chains, rings, 
and branching networks to form, 
along with other kinds of atoms 
(hydrogen, oxygen, nitrogen, and 
sulfur), a variety of structures, 
including synthetic polymers, oils, 
and the large molecules essential to 
life. 



* See textbox on p. 28 for more information on the unique properties of water. 
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Exhibit 8. Physical Science content statements for grades 4, 8, and 12 (cont.) 



Grade 4 Grade 8 Grade 12 



Energy 



Forms of Energy: Examples of forms of energy (4); kinetic energy, potential energy, and light energy from 
the Sun (8); nuclear energy and waves (12), 



P4.7: Heat (thermal energy), 
electricity, light, and sound are forms 
of energy.^ 

P4.8: Heat (thermal energy) results 
when substances burn, when certain 
kinds of materials rub against each 
other, and when electricity flows 
though wires. Metals are good 
conductors of heat (thermal energy) 
and electricity. Increasing the 
temperature of any substance requires 
the addition of energy. 

P4.9: Light travels in straight lines. 
When light strikes substances and 
objects through which it cannot pass, 
shadows result. When light travels 
obliquely from one substance to 
another ( air and water), it changes 
direction. 

P4.10: Vibrating objects produce 
sound. The pitch of sound can be 
varied by changing the rate of 
vibration. 



P8.8: Objects and substances in 
motion have kinetic energy. For 
example, a moving baseball can break 
a window; water flowing down a 
stream moves pebbles and floating 
objects along with it. 

P8.9: Three forms of potential energy 
are gravitational, elastic, and 
chemical. Gravitational potential 
energy changes in a system as the 
relative positions of objects are 
changed. Objects can have elastic 
potential energy due to their 
compression, or chemical potential 
energy due to the nature and 
arrangement of the atoms. 

P8.10: Energy is transferred from 
place to place. Light energy from the 
Sun travels through space to Earth 
(radiation). Thermal energy travels 
from a flame through the metal of a 
cooking pan to the water in the pan 
(conduction). Air warmed by a 
fireplace moves around a room 
(convection). Waves (including sound 
and seismic waves, waves on water, 
and light waves) have energy and 
transfer energy when they interact 
with matter. 

P8.ll: A tiny fraction of the light 
energy from the Sun reaches Earth. 
Light energy from the Sun is Earth's 
primary source of energy, heating 
Earth surfaces and providing the 
energy that results in wind, ocean 
currents, and storms. 



P12.8: Atoms and molecules that 
compose matter are in constant 
motion (translational, rotational, or 
vibrational). 

P12.9: Energy may be transferred 
from one object to another during 
collisions. 

P12.10: Electromagnetic waves are 
produced by changing the motion of 
charges or by changing magnetic 
fields. The energy of electromagnetic 
waves is transferred to matter in 
packets. The energy content of the 
packets is directly proportional to the 
frequency of the electromagnetic 
waves. 

P12.ll: Fission and fusion are 
reactions involving changes in the 
nuclei of atoms. Fission is the splitting 
of a large nucleus into smaller nuclei 
and particles. Fusion involves joining 
two relatively light nuclei at 
extremely high temperature and 
pressure. Fusion is the process 
responsible for the energy of the Sun 
and other stars. 



® See footnote 5 on p. 29 for more information on the framework’s use of the terms “heat” and “thermal 
energy.” 
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Exhibit 8. Physical Science content statements for grades 4, 8, and 12 (cont.) 



Grade 4 


Grade 8 


Grade 12 


Energy 


Energy Transfer and Conservatioi 

translational, rotational, and vibratio 
(12). 

i 


a: Electrical circuits (4); energy transi 
nal energy of atoms and molecules, at 

i i 


er and conservation of energy (8); 
id chemical and nuclear reactions 



P4.ll: Electricity flowing through an 
electrical circuit produces magnetic 
effects in the wires. In an electrical 
circuit containing a battery, a bulb, 
and a bell, energy from the battery is 
transferred to the bulb and the bell, 
which in turn transfer the energy to 
their surroundings as light, sound, and 
heat (thermal energy). 



P8.12: When energy is transferred 
from one system to another, the 
quantity of energy before transfer 
equals the quantity of energy after 
transfer. For example, as an object 
falls, its potential energy decreases as 
its speed, and consequently, its kinetic 
energy increases. While an object is 
falling, some of the object’s kinetic 
energy is transferred to the medium 
through which it falls, setting the 
medium into motion and heating it. 

P8.13: Nuclear reactions take place in 
the Sun. In plants, light from the Sun 
is transferred to oxygen and carbon 
compounds, which, in combination, 
have chemical potential energy 
(photosynthesis). 



P12.12: Heating increases the 
translational, rotational, and 
vibrational energy of the atoms 
composing elements and the 
molecules or ions composing 
compounds. As the translational 
energy of the atoms, molecules, or 
ions increases, the temperature of the 
matter increases. Heating a sample of 
a crystalline solid increases the 
vibrational energy of the atoms, 
molecules, or ions. When the 
vibrational energy becomes great 
enough, the crystalline structure 
breaks down and the solid melts. 

P12.13: The potential energy of an 
object on Earth’s surface is increased 
when the object’s position is changed 
from one closer to Earth’s surface to 
one farther from Earth’s surface. 

P12.14: Chemical reactions either 
release energy to the environment 
(exothermic) or absorb energy from 
the environment (endothermic). 

P12.15: Nuclear reactions (fission and 
fusion) convert very small amounts of 
matter into appreciable amounts of 
energy. 

P12.16: Total energy is conserved in a 
closed system. 
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Exhibit 8. Physical Science content statements for grades 4, 8, and 12 (cont.) 



Grade 4 



Grade 8 



Grade 12 



Motion 



Motion at the Macroscopic Level: Descriptions of position and motion (4); speed as a quantitative 
description of motion and graphical representations of speed (8); velocity and acceleration as quantitative 
descriptions of motion and the representation of linear velocity and acceleration in tables and graphs (12). 



P4.12: An object’s position can be 
described by locating the object 
relative to other objects or a 
background. The description of an 
object’s motion from one observer’s 
view may be different from that 
reported from a different observer’ s 



P4.13: An object is in motion when 
its position is changing. The speed of 
an object is defined by how far it 
travels divided by the amount of time 
it took to travel that far. 



P8.14: An object’s motion can be 
described by its speed and the 
direction in which it is moving. An 
object’s position can be measured and 
graphed as a function of time. An 
object’s speed can be measured and 
graphed as a function of time. 



P12.17: The motion of an object can 
be described by its position and 
velocity as functions of time and by 
its average speed and average 
acceleration during intervals of time. 

P12.18: Objects undergo different 
kinds of motion (translational, 
rotational, and vibrational). 
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Exhibit 8. Physical Science content statements for grades 4, 8, and 12 (cont.) 



Grade 4 Grade 8 Grade 12 



Motion 



Forces Affecting Motion: The association of changes in motion with forces and the association of objects 
falling toward Earth with gravitational force (4); qualitative descriptions of magnitude and direction as 
characteristics of forces, addition of forces, contact forces, forces that act at a distance, and net force on an 
object and its relationship to the object’s motion (8); quantitative descriptions of universal gravitational and 
electric forces, and relationships among force, mass, and acceleration (12). 



P4.14: The motion of objects can be 
changed by pushing or pulling. The 
size of the change is related to the size 
of the force (push or pull) and the 
weight (mass) of the object on which 
the force is exerted. When an object 
does not move in response to a push 
or a pull, it is because another push or 
pull (friction) is being applied by the 
environment. 

P4.15: Earth pulls down on all objects 
with a force called gravity. With a few 
exceptions (helium-filled balloons), 
objects fall to the ground no matter 
where the object is on Earth. 



P8.15: Some forces between objects 
act when the objects are in direct 
contact or when they are not touching. 
Magnetic, electrical, and gravitational 
forces can act at a distance. 

P8.16: Forces have magnitude and 
direction. Forces can be added. The 
net force on an object is the sum of all 
the forces acting on the object. A 
nonzero net force on an object 
changes the object’s motion; that is, 
the object’s speed and/or direction of 
motion changes. A net force of zero 
on an object does not change the 
object’s motion; that is, the object 
remains at rest or continues to move at 
a constant speed in a straight line. 



P12.19: The motion of an object 
changes only when a net force is 
applied. 

P12.20: The magnitude of 
acceleration of an object depends 
directly on the strength of the net 
force and inversely on the mass of the 
object. This relationship (a=F net /m) is 
independent of the nature of the force. 

P12.21: Whenever one object exerts 
force on another, a force equal in 
magnitude and opposite in direction is 
exerted by the second object back on 
the first object. In closed systems, 
momentum is the quantity of motion 
that is conserved. Conservation of 
momentum can be used to help 
validate the relationship a=F net /m. 

P12.22: Gravitation is a universal 
attractive force that each mass exerts 
on any other mass. The strength of the 
gravitational force between two 
masses is proportional to the masses 
and inversely proportional to the 
square of the distance between them. 

P12.23: Electric force is a universal 
force that exists between any two 
charged objects. Opposite charges 
attract while like charges repel. The 
strength of the electric force is 
proportional to the magnitudes of the 
charges and inversely proportional to 
the square of the distance between 
them. Between any two charged 
particles, the electric force is vastly 
greater than the gravitational force. 
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Life Science 



Life Science principles are essential for understanding the functioning of living organ- 
isms and their interactions with their environment. In addition. Life Science principles are 
crucial for understanding advances in science and technology and appreciating their im- 
plications for social and personal decisions. For example, the following discoveries of the 
past 25 years rely on understanding basic ideas in life science: publication of the human 
genome and genomes of other organisms, ability to monitor the oxygen level of specific 
regions of the brain, and depletion of the ozone layer by human activities. The media 
regularly ask questions related to health and disease, such as what constitutes a healthy 
lifestyle and how to deal with the mutability of bacteria, viruses, and parasites that thwart 
efforts to develop antibiotics and vaccines. Although science does not currently provide 
complete answers to these types of questions, it provides the tools for understanding and 
addressing them. 

Understanding principles in Life Science is inextricably linked with understanding prin- 
ciples in Physical Science and Earth and Space Sciences. “Living organisms are made of 
the same components as all other matter, involve the same kind of transformations of en- 
ergy, and move using the same basic kinds of forces” (AAAS 1994, p. 59). 

Understanding living systems and their interactions with their environment requires not 
only an understanding of various levels of biological organization — molecules, cells, tis- 
sues/organs, organisms, populations, ecosystems — but also an understanding of interac- 
tions (including the transfer of information) within and across these levels and how they 
can change over time. For example, understanding how populations of organisms change 
over time is greatly facilitated by understanding the changes that occur in DNA mole- 
cules. These changes are manifest in an organism’s traits and may affect its ability to sur- 
vive and reproduce, which can lead to changing proportions of traits in populations over 
time. 

As summarized in exhibit 9, the Life Science content statements are sorted into topics 
and subtopics that, collectively, address structure, function, and patterns of change in liv- 
ing systems. However, any attempt to organize Life Science by a linear set of topics and 
subtopics, such as those listed below, is somewhat arbitrary. The overlap is evident in 
exhibit 10, Life Science Content Statements for Grades 4, 8, and 12. 
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Exhibit 9. Life Science content topics and subtopics 





Structures and Functions of Living Systems 






Organization and development 
Matter and energy transformations 
Interdependence 






Changes in Living Systems 






Heredity and reproduction 
Evolution and diversity 







STRUCTURES AND FUNCTIONS OF LIVING SYSTEMS 

This topic comprises the ways in which living systems are organized and how living 
systems carry out their life functions. Reasoning about living systems often involves 
relating different levels of organization, from the molecule to the biosphere, and 
understanding how living systems are structured at each level. The functions of living 
systems at these levels, particularly how they transform matter and energy, are included, 
as are the interactions among living systems and how they depend on one another to carry 
out their functions. 



Organization and Development 

As it was pointed out early in the 20th century, “Long ago it became evident that the key 
to every biological problem must finally be sought in the cell; for every living organism 
is, or at some time has been, a cell” (Wilson 1928, p. 1). All living things are made up of 
cells whose work is carried out by many different types of molecules. Cellular and mo- 
lecular biology has the power to explain a wide variety of phenomena related to the or- 
ganization and development of living systems, such as synthesis and reproduction, the 
extraction of energy from food, and regulation. Living organisms have a variety of ob- 
servable features that enable them to obtain food and reproduce (grade 4). The functions 
of living organisms are carried out at different levels of organization. In multicellular or- 
ganisms, cells form organs and organ systems (grade 8). Organisms are subsystems of 
populations, communities, ecosystems, and the biosphere. Cellular processes are carried 
out by molecules, particularly proteins. These processes are regulated, both internally and 
externally, by the environments in which cells exist, including local environments that 
lead to cell differentiation during the development of multicellular organisms (grade 12). 
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Matter and Energy Transformations 

Matter and energy transformations are involved in all life processes, such as photosynthe- 
sis, growth and repair, cellular respiration, and the need of living systems for continual 
input of energy. 

All single-celled and multicellular organisms have the same basic needs: water, air, a 
source of energy and materials for growth and repair, waste disposal, and conditions for 
growth and reproduction (grade 4). In terms of matter and energy transformations, the 
source of food is the distinguishing difference between plants and animals (see textbox 
below). 



Clarification: Food 

Both plants and animals require a source of energy and materials for growth and repair, 
and both plants and animals use high-energy compounds as a source of fuel and 
building material. Plants are distinguished from animals by the fact that plants have the 
capability (through photosynthesis) to take energy from light to form higher energy 
molecules containing carbon, hydrogen, and oxygen (carbohydrates) from lower 
energy molecules. 

Plants are similar to animals in that, to make other molecules for their growth and 
reproduction, they use the energy that is released as carbohydrates react with oxygen. 

In making these other molecules, plants use the breakdown products of carbohydrates, 
along with minerals from the soil and from fertilizers (known colloquially as “plant 
foods”), as building blocks. Plants also synthesize substances (carbohydrates, fats, 
proteins, vitamins) that are components of foods eaten by animals. 

Therefore, although synthesis and breakdown are common to both plants and animals, 
photosynthesis (the conversion of light energy into stored chemical energy) is unique 
to plants, making them the primary source of energy for all animals. 



Basic needs are connected with the processes of growth and metabolism. Organisms are 
made up of carbon-containing molecules; these molecules originate in molecules that 
plants assemble from carbon dioxide and water. In converting carbon-containing mole- 
cules back to water and carbon dioxide, organisms release energy, making some of it 
available to support life functions (grade 8). Matter and energy transformations in cells, 
organisms, and ecosystems have a chemical basis (grade 12). The following textbox on 
the flow of energy through an ecosystem illustrates principles that cut across the content 
areas. 
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Crosscutting Content: Uses, Transformations, and Conservation of Energy 

The principles of energy uses, transformations, and conservation hold true across different types 
of systems. These systems include biological organisms, Earth systems, ecosystems (combining 
both life forms and their physical environment), the solar system, other systems in the universe, 
and human-designed systems. 

However complex the workings of living organisms, they share with all other 
natural systems the same physical principles of the conservation and 
transformation of matter and energy. Over long spans of time, matter and energy 
are transformed among living things, and between them and the physical 
environment. In these grand-scale cycles, the total amount of matter and energy 
remains constant, even though their form and location undergo continual change. 

Almost all life on earth is ultimately maintained by transformations of energy 
from the Sun. Plants capture the Sun’s energy and use it to synthesize complex, 
energy-rich molecules (chiefly sugars) from molecules of carbon dioxide and 
water. These synthesized molecules then serve, directly or indirectly, as the 
source of energy for the plants themselves and ultimately for all animals and 
decomposer organisms (such as bacteria and fungi). This is the food web: The 
organisms that consume the plants derive energy and materials from breaking 
down the plant molecules, use them to synthesize their own structures, and then 
arc themselves consumed by other organisms. At each stage in the food web, 
some energy is stored in newly synthesized structures and some is dissipated into 
the environment as heat produced by the energy-releasing chemical processes in 
cells. A similar energy cycle begins in the oceans with the capture of the Sun’s 
energy by tiny, plant-like organisms. Each successive stage in a food web 
captures only a small fraction of the energy content of organisms it feeds on. 

Science for All Americans (AAAS 1994, p. 66). 

The flow of energy in an ecosystem (such as that described above) can be compared to the flow 
of energy illustrated earlier (see “Crosscutting Content: Energy Sources and Transfer” textbox on 
p. 31). They are both identical (the principle) and different (the context). In each case, energy is 
transformed from one form to another; while some is no longer available for human use, it is not 
lost to the system. 



The following grade 4 content statements illustrate the crosscutting nature of uses, 
transformations, and conservation of energy. They are not intended to represent an exhaustive 
catalog of all statements related to this crosscutting content. 



Physical Science 


Life Science 


Earth and Space Sciences 


P4.7: Heat (thermal energy), 
electricity, light, and sound are 
forms of energy. 


L4.2: Organisms have basic 
needs. Animals require air, 
water, and a source of energy 
and building material for growth 
and repair. Plants also reauire 
light. 


E4.7: The Sun warms the land, 
air, and water and helps plants 
grow. 
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Interdependence 

The interaction of species in an ecosystem, the dynamics of population growth and de- 
cline, the use of resources by multiple species, the impact of species on their environ- 
ment, and the complex interactions among all of these forces have enormous conse- 
quences for the survival of all species, including humans. 

All animals and most plants depend on both other organisms and their environments for 
their basic needs (grade 4). Organisms interact with one another in a variety of ways, 
such as producer/consumer, predator/prey, and parasite/host. In addition to competition 
among organisms, the size of populations depends on environmental conditions such as 
the availability of water, light, and other suitable conditions (grade 8). Ecosystems are 
characterized by both stability and change, on which human populations can have an im- 
pact (grade 12). 

CHANGES IN LIVING SYSTEMS 

This topic comprises how organisms reproduce, how they pass genetic information to 
their offspring, and how genetic information can change as it passes from one generation 
to the next. Over time, these changes can affect the size, diversity, and genetic composi- 
tion of populations (i.e., the process of biological evolution). 

Heredity and Reproduction 

Organisms closely resemble their parents; their slight variations can accumulate over 
many generations and result in more obvious differences between organisms and their 
ancestors. Recent advances in biochemistry and cell biology have increased understand- 
ing of the mechanisms of inheritance and have enabled the detection of disease-related 
genes. Such knowledge is making it possible to design and produce large quantities of 
substances to treat disease and, in years to come, may lead to cures. 

All plants and animals (and one-celled organisms) develop and have the capacity to re- 
produce (grade 4). Reproduction, whether sexual or asexual, is a requirement for the sur- 
vival of a species. Characteristics of organisms are influenced by heredity and environ- 
ment (grade 8). Genetic differences among individuals and species are fundamentally 
chemical. Different organisms are made up of somewhat different proteins. Reproduction 
involves passing the DNA with instructions for making these proteins from one genera- 
tion to the next with occasional modifications (grade 12). 

Evolution and Diversity 

Earth’s present-day life forms have evolved from common ancestors reaching back to the 
simplest one-celled organisms almost 4 billion years ago. Modern ideas about evolution 
provide a scientific explanation for three main sets of observable facts about life on 
Earth: the enormous number of different life forms that exist, the systematic similarities 
in anatomy and molecular chemistry seen within that diversity, and the sequences of 
changes in fossils found in successive layers of rock that have been formed over more 
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than a billion years. The modem concept of evolution, including natural selection and 
common descent, provides a unifying principle for understanding the history of life on 
Earth, relationships among all living things, and the dependence of life on the physical 
environment. The concept is so well established that it provides a framework for organiz- 
ing most of biological knowledge into a coherent picture. 

All organisms are similar to and different from other organisms, and some kinds of or- 
ganisms and individuals have advantages in particular environments (grade 4). Preferen- 
tial survival means that differences among individuals in a population affect their ability 
to survive and reproduce. Classification reflects degrees of relatedness among species 
(grade 8). Evolution is the consequence of natural selection and differential reproduction. 
Natural selection and common descent provide the scientific explanation for the history 
of life on Earth as depicted in the fossil record and as indicated by anatomical and chemi- 
cal similarities evident within the diversity of existing organisms (grade 12). 
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Exhibit 10. Life Science content statements for grades 4, 8, and 12 



Grade 4 


Grade 8 


Grade 12 


Structures and Functions of Living Systems 


Organization and Development 

systems (8) the chemical basis of 


: Basic needs of organisms (4), lev 
living systems (12). 


'els of organization of living 



L4.1: Organisms need food, water, 
and air; a way to dispose of waste; 
and an environment in which they can 
live.* 



L8.1: All organisms are composed of 
cells, from one cell only to many 
cells. About two-thirds of the weight 
of cells is accounted for by water, 
which gives cells many of their 
properties. In multicellular organisms, 
specialized cells perform specialized 
functions. Organs and organ systems 
are composed of cells and function to 
serve the needs of cells for food, air, 
and waste removal. The way in which 
cells function is similar in all living 
organisms. ^ 

L8.2: Following fertilization, cell 
division produces a small cluster of 
cells that then differentiate by 
appearance and function to form the 
basic tissues of an embryo. 



L12.1: Living systems are made of 
complex molecules (including 
carbohydrates, fats, proteins, and 
nucleic acids ) that consist mostly of a 
few elements, especially carbon, 
hydrogen, oxygen, nitrogen, and 
phosphorous. 

L12.2: Cellular processes are carried 
out by many different types of 
molecules, mostly proteins. Protein 
molecules are long, usually folded 
chains made from combinations of 
amino-acid molecules. Protein 
molecules assemble fats and 
carbohydrates and carry out other 
cellular functions. The function of 
each protein molecule depends on its 
specific sequence of amino acids and 
the shape of the molecule. 

L12.3; Cellular processes are 
regulated both internally and 
externally by environments in which 
cells exist, including local 
environments that lead to cell 
differentiation during the development 
of multicellular organisms. During the 
development of complex multicellular 
organisms, cell differentiation is 
regulated through the expression of 
different genes. 



* See p. 41 for a textbox on “Food.” 

' Fluman organs and organ systems are subsumed under this content statement. See the specifications for 
more information. 
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Exhibit 10. Life Science content statements for grades 4, 8, and 12 (cont.) 



Grade 4 


Grade 8 


Grade 12 


Structures and Functions of Living Systems 


Matter and Energy Transforms 

carbon compounds in growth and 
transformation in living systems ( 


itions: The basic needs of organisi 
metabolism (8), the chemical basi 
12). 


ns for growth (4), the role of 
s of matter and energy 



L4.2: Organisms have basic needs. 
Animals require air, water, and a 
source of energy and building material 
for growth and repair. Plants also 
require light. 



L8.3: Cells carry out the many 
functions needed to sustain life. They 
grow and divide, thereby producing 
more cells. Food is used to provide 
energy for the work that cells do and 
is a source of the molecular building 
blocks from which needed materials 
are assembled. 

L8.4: Plants are producers; that is, 
they use the energy from light to make 
sugar molecules from the atoms of 
carbon dioxide and water. * Plants use 
these sugars along with minerals from 
the soil to form fats, proteins, and 
carbohydrates. These products can be 
used immediately, incorporated into 
the plant’s cells as the plant grows, or 
stored for later use. 

L8.5: All animals, including humans, 
are consumers that meet their energy 
needs by eating other organisms or 
their products. Consumers break down 
the structures of the organisms they 
eat to make the materials they need to 
grow and function. Decomposers, 
including bacteria and fungi, use dead 
organisms or their products to meet 
their energy needs. 



L12.4: Plants have the capability 
(through photosynthesis) to take 
energy from light to form higher 
energy sugar molecules containing 
carbon, hydrogen, and oxygen from 
lower energy molecules. These sugar 
molecules can be used to make amino 
acids and other carbon-containing 
(organic) molecules and assembled 
into larger molecules with biological 
activity (including proteins, DNA, 
carbohydrates, and fats). 

L12.5: The chemical elements that 
make up the molecules of living 
things pass through food webs and are 
combined and recombined in different 
ways. At each link in an ecosystem, 
some energy is stored in newly made 
structures, but much is dissipated into 
the environment as heat. Continual 
input of energy from sunlight keeps 
the process going. 

L12.6: As matter cycles and energy 
flows through different levels of 
organization of living systems (cells, 
organs, organisms, communities) and 
between living systems and the 
physical environment, chemical 
elements are recombined in different 
ways. Each recombination results in 
storage and dissipation of energy into 
the environment as heat. Matter and 
energy are conserved in each change. 



* The statement “they use the energy from light” does not imply that energy is converted into matter or that 
energy is lost. See textbox “Crosscutting Content: Uses, Transformations, and Conservation of Energy.” 
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Exhibit 10. Life Science content statements for grades 4, 8, and 12 (cont.) 



Grade 4 Grade 8 Grade 12 



Structures and Functions of Living Systems 



Interdependence: The interdependence of organisms (4), specific types of interdependence (8), 
consequences of interdependence (12). 



L4.3: Organisms interact and are 
interdependent in various ways, 
including providing food and shelter 
to one another. Organisms can survive 
only in environments in which their 
needs are met. Some interactions are 
beneficial; others are detrimental to 
the organism and other organisms. 

L4.4: When the environment changes, 
some plants and animals survive and 
reproduce; others die or move to new 
locations. 



L8.6: Two types of organisms may 
interact with one another in several 
ways: They may be in a 
producer/consumer, predator/prey, or 
parasite/host relationship. Or, one 
organism may scavenge or decompose 
another. Relationships may be 
competitive or mutually beneficial. 
Some species have become so adapted 
to each other that neither could 
survive without the other. 

L8.7: The number of organisms and 
populations an ecosystem can support 
depends on the biotic resources 
available and abiotic factors, such as 
quantity of light and water, range of 
temperatures, and soil composition. 

L8.8: All organisms cause changes in 
the environment where they live. 

Some of these changes are detrimental 
to the organisms or other organisms, 
whereas others are beneficial. 



L12.7: Although the interrelationships 
and interdependence of organisms 
may generate biological communities 
in ecosystems that are stable for 
hundreds or thousands of years, 
ecosystems always change when 
climate changes or when one or more 
new species appear as a result of 
migration or local evolution. The 
impact of the human species has 
major consequences for other species. 
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Exhibit 10. Life Science content statements for grades 4, 8, and 12 (cont.) 



Grade 4 


Grade 8 


Grade 12 


Changes in Living Systems 


Heredity and Reproduction: Li 

environment on an offspring’s eh 


fe cycles (4), reproduction and the 
aracteristics (8), the molecular has 


influence of heredity and the 
is of heredity (12). 



L4.5: Plants and animals have life 
cycles. Both plants and animals begin 
life and develop into adults, 
reproduce, and eventually die. The 
details of this life cycle are different 
for different organisms. 

L4.6: Plants and animals closely 
resemble their parents. 



L8.9: Reproduction is a characteristic 
of all living systems; because no 
individual organism lives forever, 
reproduction is essential to the 
continuation of every species. Some 
organisms reproduce asexually. Other 
organisms reproduce sexually. 

L8.10: The characteristics of 
organisms are influenced by heredity 
and environment. For some 
characteristics, inheritance is more 
important; for other characteristics, 
interactions with the environment are 
more important. 



L12.8: Hereditary information is 
contained in genes, which are located 
in the chromosomes of each cell. A 
human cell contains many thousands 
of different genes. One or many genes 
can determine an inherited trait of an 
individual, and a single gene can 
influence more than one trait. 

L12.9; The genetic information 
encoded in DNA molecules provides 
instructions for assembling protein 
molecules. Genes are segments of 
DNA molecules. Inserting, deleting, 
or substituting DNA segments can 
alter genes. An altered gene may be 
passed on to every cell that develops 
from it. The resulting features may 
help, harm, or have little or no effect 
on the offspring’s success in its 
environment. 

L12.10: Sorting and recombination of 
genes in sexual reproduction results in 
a great variety of possible gene 
combinations from the offspring of 
any two parents. 



NAEP 2009 Science Framework 



48 






Exhibit 10. Life Science content statements for grades 4, 8, and 12 (cont.) 



Grade 4 


Grade 8 


Grade 12 


Changes in Living Systems 


Evolution and Diversity: Diffen 
relatedness of organisms (8), the 
(12). 


;nces and adaptations of organisms 
mechanisms of evolutionary chang 


i (4), preferential survival and 
;e and the history of life on Earth 



L4.7: Different kinds of organisms 
have characteristics that enable them 
to survive in different environments. 
Individuals of the same kind differ in 
their characteristics, and sometimes 
the differences give individuals an 
advantage in surviving and 
reproducing. 



L8.ll: Individual organisms with 
certain traits in particular 
environments are more likely than 
others to survive and have offspring. 
When an environment changes, the 
advantage or disadvantage of 
characteristics can change. Extinction 
of a species occurs when the 
environment changes and the 
characteristics of a species are 
insufficient to allow survival. Fossils 
indicate that many organisms that 
lived long ago are extinct. Extinction 
of a species is common; most of the 
species that have lived on the Earth no 
longer exist. 

L8.12: Similarities among organisms 
are found in anatomical features, 
which can be used to infer the degree 
of relatedness among organisms. In 
classifying organisms, biologists 
consider details of internal and 
external structures to be more 
important than behavior or general 
appearance. 



L12.ll: Modern ideas about evolution 
( including natural selection and 
common descent) provide a scientific 
explanation for the history of life on 
Earth as depicted in the fossil record 
and in the similarities evident within 
the diversity of existing organisms. 

L12.12: Molecular evidence 
substantiates the anatomical evidence 
for evolution and provides additional 
detail about the sequence in which 
various lines of descent branched. 

L12.13: Evolution is the consequence 
of the interactions of (1) the potential 
for a species to increase its numbers, 
(2) the genetic variability of offspring 
due to mutation and recombination of 
genes, (3) a finite supply of the 
resources required for life, and (4) the 
ensuing selection from environmental 
pressure of those organisms better 
able to survive and leave offspring. 
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Earth and Space Sciences 



The past few decades have brought rapid changes in the character of Earth and Space 
Sciences. The study of Earth has shifted from surface geology and mining toward global 
change and Earth systems; research methods have changed from human observations and 
mapping to remote sensing and computer modeling. 6 This concept of Earth as a complex 
and dynamic entity of interrelated subsystems implies that there is no process or phe- 
nomenon within the Earth system that occurs in complete isolation from other elements 
of the system. There has also been a shift in goals as advances in theory have made it 
possible to more accurately predict changes in weather and climate; to provide life-saving 
warnings of floods, hurricanes, earthquakes, and volcanic eruptions; and to understand 
how human activities influence ecosystem and climate changes across the globe. 

In space science, similar changes have taken place as a result of new technologies. Suc- 
cessful probes to Mars, Jupiter, and Saturn have vastly expanded knowledge of the solar 
neighborhood. The discovery of more than 100 planets outside the solar system has raised 
new questions about the origin of life. Furthermore, advances in ground and space-based 
telescopes capable of observing many different parts of the electromagnetic spectrum 
with unprecedented detail have revolutionized understanding of the structure and evolu- 
tion of the universe itself. In brief, descriptive methods of Earth and Space Sciences have 
given way to theory-based inquiry and problem-solving approaches that have far- 
reaching consequences with regard to understanding the universe and stewardship of 
planet Earth. 

Changes in Earth and Space Science education are beginning to catch up with advances 
in research. The National Standards emphasize a systems approach to studying Earth, 
especially at the high school level. Some of today’s textbooks pay less attention to de- 
scribing Earth features and focus instead on a systems perspective in which Earth is 
viewed as a physical system of interrelated phenomena, processes, and cycles. Some high 
school curricula have integrated the traditional Earth science disciplines of geology, 
meteorology, and oceanography with aspects of biology, chemistry, and physics to intro- 
duce students to a more holistic study of Earth. 

The tools available to students for learning about Earth and space have changed as well, 
although not all of these resources are available to all students. Visualization tools, such 
as geographical information system (GIS) software, have made it possible for Earth sci- 
ence students to have direct access to the raw data and models used by scientists. Other 
Web-based programs allow students to view and process satellite images of Earth, to di- 
rect a camera on board the Space Shuttle, and to access professional telescopes around 
the world to carry out science projects. In other words, the core concepts, subject matter, 
and tools used by students have undergone profound changes in recent decades that 



6 “Earth” is capitalized, rather than referred to as “the earth,” in order to recognize it as one of the planets in 
the solar system. 
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mirror many of the advances in Earth and Space Sciences. The data and images gathered 
by these tools could be used as source materials for assessment items. For example, stu- 
dents could examine data on sea surface temperatures and upper atmospheric winds, de- 
rived from satellite observations, to predict the intensity and track of a hurricane. 

To reflect the importance of this content area, NAEP will include questions about Earth 
and Space Sciences at the 4th, 8th, and 12th grade levels. The content statements have 
been divided into topics and subtopics as summarized in exhibit 11. 

Exhibit 11. Earth and Space Sciences content topics and subtopics 



Earth in Space and Time 

Objects in the universe 
History of Earth 

Earth Structures 



Properties of Earth materials 
Tectonics 



Earth Systems 



Energy in Earth systems 
Climate and weather 
Biogeochemical cycles 



EARTH IN SPACE AND TIME 

Earth in space and time is divided into two subtopics: objects in the universe and history 
of Earth. The idea that “the universe is large and ancient, on scales staggering to the hu- 
man mind” (AAAS 1994, p. 40) connects these subtopics. 

One of the earliest discoveries of the scientific age was that Earth is not the center of the 
universe. It is now known that Earth is a planet in space, one of a family of planets and 
other bodies that circle a yellow star in a vast galaxy of other stars. Like countless other 
worlds that are known to exist, Earth has a beginning and a history. That history can be 
read by carefully and thoughtfully observing the world and the universe. 

Objects in the Universe 

Objects in the sky, such as the Sun and the Moon, have patterns of movement. These 
patterns can be observed through changes in shape or placement in the sky based on time 
of day or season (grade 4). By recognizing these patterns, people have developed 
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calendars and clocks and explained such phenomena as Moon phases, eclipses, and 
seasons (grade 8). 

It was previously thought that Earth was the center of the universe. However, it is now 
known that the Sun is the central and largest body in the solar system, which includes 
Earth and other planets and their moons as well as other objects such as asteroids and 
comets. Objects in the solar system are kept in predictable motion by the force of gravity 
(grade 8). 

According to the “big bang” theory, the entire contents of the known universe expanded 
explosively into existence from a hot, dense state 13.7 billion years ago. Early in the his- 
tory of the universe, stars coalesced out of clouds of hydrogen and helium and clumped 
together by gravitational attraction into billions of galaxies. When heated to a sufficiently 
high temperature by gravitational attraction, stars begin nuclear reactions, which convert 
matter to energy and fuse light elements into heavier ones (grade 12). 

History of Earth 

Theories of planet formation and radioactive dating of meteorites have led to the conclu- 
sion that the Sun, Earth, and the rest of the solar system formed from a nebular cloud of 
dust and gas 4.6 billion years ago. Early Earth was very different from today’s planet. 
Initially, there was no life and no molecular oxygen in the atmosphere. Evidence shows 
that one-celled organisms (bacteria) were the first forms of life on the planet, appearing 
about 3.5 billion years ago. These bacteria are thought to be responsible for adding oxy- 
gen to Earth’s atmosphere, making it possible for a wider diversity of life forms to evolve 
(grade 12). 

Earth processes seen today, such as erosion and mountain building, have made possible 
the measurement of geologic time through methods such as observing rock sequences and 
using fossils to correlate the sequences at various locations. Fossils also provide evidence 
of how life and environmental conditions have changed (grade 8). Early methods of de- 
termining geological time, such as the use of index fossils and stratigraphic sequences, 
allowed for the relative dating of geologic events. However, absolute dating was impos- 
sible until the discovery that certain radioactive isotopes in rocks have known decay 
rates, making it possible to determine how many years ago a given rock sample formed 
(grade 12). 

Earth’s surface changes over time. Some changes are due to slow processes, such as ero- 
sion and weathering, and others are due to rapid processes such as volcanic eruptions, 
landslides, and earthquakes (grade 4). Changes caused by violent earthquakes and vol- 
canic eruptions can be observed on a human time scale; on the other hand, many geologi- 
cal processes, such as the building of mountain chains and shifting of entire continents, 
take place over hundreds of millions of years. Water, ice, waves, and wind sculpt Earth’s 
surface to produce distinctive landforms (grade 12). 
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EARTH STRUCTURES 



Content statements related to Earth structures are divided into two subtopics: properties 
of Earth materials and tectonics. The study of Earth materials has contributed to under- 
standing dynamic processes, which are in turn driven by the movement of vast tectonic 
plates. Conversely, the development of tectonic theory has made it possible to locate and 
extract Earth materials for a wide variety of human uses. 

Properties of Earth Materials 

Earth materials that occur in nature include rocks, minerals, soils, water, and the gases of 
the atmosphere. Natural materials have different properties that sustain plant and animal 
life (grade 4). Soil consists of weathered rocks and decomposed organic material from 
dead plants, animals, and bacteria. Soils are often found in layers, with each having a dif- 
ferent chemical composition and texture (grade 8). Some Earth materials have properties 
(either in their present form or after design and modification) that make them useful in 
solving human problems and enhancing the quality of life (grade 4). 

Rocks and rock formations bear evidence of the conditions and forces that created them, 
ranging from the violent conditions of volcanic eruptions to the slow deposition of sedi- 
ments. The atmosphere is a mixture of nitrogen, oxygen, and trace gases that include wa- 
ter vapor. The atmosphere has a different physical and chemical composition at different 
elevations (grade 8). 

Tectonics 

A basic understanding of geological history (described above) forms the foundation for 
later understanding of tectonics. Earth’s internal structure is layered with a lithosphere, 
hot convecting mantle, and dense metallic core. Lithospheric plates, on the scale of conti- 
nents and oceans, constantly move at rates of centimeters per year in response to move- 
ments in the mantle. Major geological events such as earthquakes, volcanic eruptions, 
and mountain building result from these plate motions (grade 8). 

Although continental drift was first suggested as early as the late 16th century and further 
developed in the early 1900s, it was not widely accepted until more convincing evidence 
emerged as a result of extensive exploration of the sea floor. By the late 1960s, mapping 
of the Mid-Atlantic Ridge, evidence of sea floor spreading, and subduction led to the 
more general theory of plate tectonics. The current explanation is that the outward trans- 
fer of Earth’s internal heat propels the plates comprising Earth’s surface across the face 
of the globe, pushing the plates apart where magma rises to form midocean ridges, and 
pulling the edges of plates back down where Earth materials sink into the crust at deep 
trenches (grade 12). 

Earth as a whole has a magnetic field that is detectable at the surface with a compass. 
Earth’s magnetic field is similar to the field of a natural or manmade magnet with north 
and south poles and lines of force. For thousands of years, people have used compasses to 
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aid navigation on land and sea (grade 8). Crucial evidence in support of tectonic theory 
came from studies of the magnetic properties of rocks on the ocean floor (grade 12). 

EARTH SYSTEMS 

Earth systems is divided into three subtopics: energy in Earth systems, climate and 
weather, and biogeochemical cycles. The explorers of the 16th century who circumnavi- 
gated the planet were the first to become aware of global weather and climate patterns. 

As science began to mature and diversify in the 19th and 20th centuries, those who stud- 
ied the planet scientifically did so from the perspective of the traditional disciplines, such 
as geology, oceanography, and meteorology. Currently, working with vastly improved 
technologies, most scientists take an Earth systems perspective, including the study of 
how energy moves through Earth systems and the integration of disciplines to better un- 
derstand Earth’s biogeochemical cycles. 

Energy in Earth Systems 

The Sun warms the land, air, and water and helps plants grow (grade 4). The Sun is the 
major source of energy for phenomena on Earth’s surface. It drives convection within the 
atmosphere and oceans, producing winds, ocean currents, and the water cycle. Seasons 
result from annual variations in the intensity of sunlight and length of day due to the tilt 
of Earth’s rotation axis relative to the plane of its yearly orbit around the Sun (grade 8). 

Earth’s systems have internal and external sources of energy, both of which create heat. 
The Sun is the major external source of energy. Two primary sources of internal energy 
are the decay of radioactive isotopes and the gravitational/thermal energy from Earth’s 
original formation (grade 12). 

Climate and Weather 

Weather changes from day to day and during the seasons. Scientists use tools for re- 
cording and predicting weather changes (grade 4). Global patterns of atmospheric move- 
ment influence local weather (grade 8). 

Climate is determined by energy transfer from the Sun at and near Earth’s surface. This 
energy transfer is influenced by dynamic processes such as cloud cover, atmospheric 
gases, and Earth’s rotation, as well as static conditions such as the position of mountain 
ranges and oceans, seas, and lakes (grade 12). Oceans have a major effect on climate be- 
cause water in the oceans holds a large amount of heat (grade 8). 

Biogeochemical Cycles 

Earth is a system containing essentially a fixed amount of each stable chemical atom or 
element. Each element can exist in several different chemical forms. Elements move 
within and between the lithosphere, atmosphere, hydrosphere, and biosphere as part of 
biogeochemical cycles (see the following textbox). Movement of matter through Earth’s 
systems is driven by Earth’s internal and external sources of energy. These movements 
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are often accompanied by a change in the physical and chemical properties of the matter. 
For example, carbon occurs in carbonate rocks such as limestone, in coal and other fossil 
fuels, in the atmosphere as carbon dioxide gas, in water as dissolved carbon dioxide, and 
in all organisms as complex molecules that control the chemistry of life (grade 12). 

Water, which covers the majority of Earth’s surface, circulates through the crust, oceans, 
and atmosphere in what is known as the water cycle. Water evaporates from Earth’s sur- 
face, rises and cools as it moves to higher elevations, condenses as clouds, falls as rain or 
snow, and collects in lakes, oceans, soil, and underground (grade 8). 

Natural ecosystems provide an array of basic processes that affect humans. These proc- 
esses include maintenance of the quality of the atmosphere, generation of soils, control of 
the hydrologic cycle, disposal of wastes, and recycling of nutrients (grade 12). The sup- 
ply of many Earth resources such as fuels, metals, fresh water, and farmland is limited. 

Humans change environments in ways that can either be beneficial or detrimental for 
themselves and other organisms (grade 4). Humans have devised methods for extending 
the use of Earth resources through recycling, reuse, and renewal (grade 4). However, 
other activities (reducing the amount of forest cover, increasing the amount and variety of 
chemicals released into the atmosphere, and intensive farming) have changed Earth’s 
land, oceans, and atmosphere. Studies of plant and animal populations have shown that 
such activities can reduce the number and variety of wild plants and animals and some- 
times result in the extinction of species (grade 8). 
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Crosscutting Content: Biogeochemical Cycles 

To demonstrate an understanding of biogeochemical cycles, students must draw on their 
knowledge of matter and energy (Physical Science), structures and functions of living systems 
(Life Science) and Earth systems (Earth and Space Sciences). 

Essentially fixed amounts of chemical atoms or elements cycle within the Earth system, and 
energy drives their translocation and transformation. Examples of biogeochemical cycles include 
water, carbon, and nitrogen. The basic processes underlying the translocation of matter (e.g., 
changes of state, gravity) and transformations involving the rearrangement of atoms in chemical 
reactions are described in Physical Science (exhibit 8 on p. 34) and the role of living organisms in 
cycling atoms between inorganic and organic forms is described in Life Science (exhibit 10 on p. 
46). 

Biogeochemical cycles arc described more fully in the Earth Systems section of exhibit 12, Earth 
and Space Science Content Statements for Grades 4, 8, and 12. 



The following grade 12 content statements illustrate the crosscutting nature of biogeochemical 
cycles. They are not intended to represent an exhaustive catalog of all statements related to this 
crosscutting content. 



Physical Science 


Life Science 


Earth and Space Sciences 


P12.7: A large number of 
important reactions involve the 
transfer of either electrons 
(oxidation/reduction reactions) 
or hydrogen ions (acid/base 
reactions) between reacting ions, 
molecules, or atoms. In other 
chemical reactions, atoms 
interact with one another by 
sharing electrons to create a 
bond. An important example is 
carbon atoms, which can bond to 
one another in chains, rings, and 
branching networks to form, 
along with other kinds of atoms 
(hvdrogen, oxvgen. nitrogen, 
and sulfur) a variety of 
structures, including synthetic 
polymers, oils, and the large 
molecules essential to life. 


L12.5: The chemical elements 
that make up the molecules of 


E12.12: Movement of matter 
through Earth’s systems is 
driven by Earth’s internal and 
external sources of energy. 

These movements are often 
accompanied by a change in the 
physical and chemical properties 
of the matter. Carbon, for 
example, occurs in carbonate 
rocks such as limestone, in coal 
and other fossil fuels, in the 
atmosphere as carbon dioxide 
gas, in water as dissolved carbon 
dioxide, and in all organisms as 
complex molecules that control 
the chemistry of life. 


living things pass through food 
webs and are combined and 
recombined in different wavs. At 
each link in an ecosystem, some 
energy is stored in newly made 
structures, but much is 
dissipated into the environment 
as heat. Continual input of 
energy from sunlight keeps the 
process going. 
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Exhibit 12. Earth and Space Sciences content statements for grades 4, 8, and 12 
Grade 4 Grade 8 Grade 12 



Earth in Space and Time 



Objects in the Universe: Patterns in the sky (4), a model of the solar system (8), a vision of the 
universe (12). 



E4.1: Objects in the sky have patterns 
of movement. The Sun, for example, 
appears to move across the sky in the 
same way every day, but its path 
changes slowly over the seasons. The 
Moon appears to move across the sky 
on a daily basis much like the Sun. 

E4.2: The observable shape of the 
Moon changes from day to day in a 
cycle that lasts about a month. 



E8.1: In contrast to an earlier theory 
that Earth is the center of the universe, 
it is now known that the Sun, an 
average star, is the central and largest 
body in the solar system. Earth is the 
third planet from the Sun in a system 
that includes seven other planets and 
their moons, as well as smaller objects 
such as asteroids and comets. 

E8.2: Gravity is the force that keeps 
most objects in the solar system in 
regular and predictable motion. These 
motions explain such phenomena as 
the day, the year, phases of the Moon, 
and eclipses. 



E12.1: The origin of the universe 
remains one of the greatest questions 
in science. The “big bang” theory 
places the origin approximately 13.7 
billion years ago when the universe 
began in a hot, dense state. According 
to this theory, the universe has been 
expanding ever since. 

E12.2: Early in the history of the 
universe, matter (primarily the light 
atoms hydrogen and helium) clumped 
together by gravitational attraction to 
form countless trillions of stars and 
billions of galaxies. 

E12.3: Stars, like the Sun, transform 
matter into energy in nuclear 
reactions. When hydrogen nuclei fuse 
to form helium, a small amount of 
matter is converted to energy. These 
and other processes in stars have led 
to the formation of all the other 
elements. 
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Exhibit 12. Earth and Space Sciences content statements for grades 4, 8, and 12 (cont.) 



Grade 4 


Grade 8 


Grade 12 


Earth in Space and Time 


History of Earth: Evidence of cl 
(8), theories about Earth’s history 

i 


lange (4), estimating the timing an 
' (12). 

i i 


d sequence of geologic events 



E4.3: The surface of Earth changes. 
Some changes are due to slow 
processes such as erosion and 
weathering, and some changes are due 
to rapid processes such as landslides, 
volcanic eruptions, and earthquakes. 



E8.3: Fossils provide important 
evidence of how life and 
environmental conditions have 
changed in a given location. 

E8.4: Earth processes seen today, 
such as erosion and mountain 
building, make it possible to measure 
geologic time through methods such 
as observing rock sequences and using 
fossils to correlate the sequences at 
various locations. 



E12.4: Early methods of determining 
geologic time, such as the use of 
index fossils and stratigraphic 
sequences, allowed for the relative 
dating of geological events. However, 
absolute dating was impossible until 
the discovery that certain radioactive 
isotopes in rocks have known decay 
rates, making it possible to determine 
how many years ago a given rock 
sample formed. 

E12.5: Theories of planet formation 
and radioactive dating of meteorites 
and lunar samples have led to the 
conclusion that the Sun, Earth, and the 
rest of the solar system formed from a 
nebular cloud of dust and gas 4.6 
billion years ago. 

E12.6: Early Earth was very different 
from today’s planet. Evidence for 
one-celled forms of life (bacteria) 
extends back more than 3.5 billion 
years. The evolution of life caused 
dramatic changes in the composition 
of Earth's atmosphere, which did not 
originally contain molecular oxygen. 

E12.7: Earth's current structure has 
been influenced by both sporadic and 
gradual events. Changes caused by 
violent earthquakes and volcanic 
eruptions can be observed on a human 
time scale; however, many geological 
processes, such as the building of 
mountain chains and shifting of entire 
continents, take place over hundreds 
of millions of years. 
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Exhibit 12. Earth and Space Sciences content statements for grades 4, 8, and 12 (cont.) 



Grade 4 



Earth Structures 



Grade 8 



Grade 12 



Properties of Earth Materials: Natural and manmade materials (4), soil analysis and layers of the 
atmosphere (8). 



E4.4: Earth materials that occur in 
nature include rocks, minerals, soils, 
water, and the gases of the 
atmosphere. 

E4.5: Natural materials have different 
properties that sustain plant and 
animal life. 

E4.6: Some Earth materials have 
properties (either in their present form 
or after design and modification) that 
make them useful either in their 
present form or designed and 
modified to in solving human 
problems and enhancing the quality of 
life, as in the case of materials used 
for building or fuels used for heating 
and transportation. 



E8.5: Rocks and rock formations bear 
evidence of the minerals, materials, 
temperature/pressure conditions, and 
forces that created them. Some 
formations show evidence that they 
were deposited by volcanic eruptions. 
Others are composed of sand and 
smaller particles that are buried and 
cemented by dissolved minerals to 
form solid rock again. Still others 
show evidence that they were once 
earlier rock types that were exposed to 
heat and pressure until they changed 
shape and, in some cases, melted and 
recrystallized. 

E8.6: Soil consists of weathered rocks 
and decomposed organic material 
from dead plants, animals, and 
bacteria. Soils are often found in 
layers with each having a different 
chemical composition and texture. 

E8.7: The atmosphere is a mixture of 
nitrogen, oxygen, and trace gases that 
include water vapor. The atmosphere 
has a different physical and chemical 
composition at different elevations. 
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Exhibit 12. Earth and Space Sciences content statements for grades 4, 8, and 12 (cont.) 



Grade 4 


Grade 8 


Grade 12 


Earth Structures 


Tectonics: The basics of tectonic 
drives tectonics and its supporting 

i 


theory and Earth magnetism (8), t 
l evidence (12). 

i i 


he physical mechanism that 



E8.8: Earth is layered with a 
lithosphere; a hot, convecting mantle; 
and a dense, metallic core. 

E8.9: Lithospheric plates on the scale 
of continents and oceans constantly 
move at rates of centimeters per year 
in response to movements in the 
mantle. Major geological events, such 
as earthquakes, volcanic eruptions, 
and mountain building, result from 
these plate motions. 

E8.10: Earth as a whole has a 
magnetic field that is detectable at the 
surface with a compass. Earth’s 
magnetic field is similar to the field of 
a natural or manmade magnet with 
north and south poles and lines of 
force. For thousands of years, people 
have used compasses to aid in 
navigation on land and sea. 



E12.8: Mapping of the Mid-Atlantic 
Ridge, evidence of sea floor 
spreading, and subduction provided 
crucial evidence in support of the 
theory of plate tectonics. The theory 
currently explains plate motion as 
follows: the outward transfer of 
Earth's internal heat propels the plates 
comprising Earth's surface across the 
face of the globe. Plates are pushed 
apart where magma rises to form 
midocean ridges, and the edges of 
plates are pulled back down where 
Earth materials sink into the crust at 
deep trenches. 
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Exhibit 12. Earth and Space Sciences content statements for grades 4, 8, and 12 (cont.) 



Grade 4 


Grade 8 


Grade 12 


Earth Systems 


Energy in Earth Systems: The role of the Sun (4), the Sun’s observable effects (8), internal and 
external sources of energy in Earth systems (12). 


E4.7: The Sun warms the land, air, 
and water and helps plants grow. 


E8.ll: The Sun is the major source of 
energy for phenomena on Earth’s 
surface. It provides energy for plants 
to grow and drives convection within 
the atmosphere and oceans, producing 
winds, ocean currents, and the water 
cycle. 

E8.12: Because of the tilt of Earth’s 
rotation axis relative to the plane of its 
yearly orbit around the Sun, seasons 
result from annual variations in the 
intensity of sunlight and length of day. 
the tilt of Earth's rotation axis relative 
to the plane of its yearly orbit around 
the sun. 


E12.9: Earth systems have internal 
and external sources of energy, both 
of which create heat. The Sun is the 
major external source of energy. Two 
primary sources of internal energy are 
the decay of radioactive isotopes and 
the gravitational energy from Earth's 
original formation. 


Climate and Weather: Local weather (4), global weather patterns (8), systems that influence 
climate (12). 


E4.8: Weather changes from day to 
day and during the seasons. 

E4.9: Scientists use tools for 
observing, recording, and predicting 
weather changes from day to day and 
during the seasons. 


E8.13: Global patterns of atmospheric 
movement influence local weather. 
Oceans have a major effect on climate 
because water in the oceans holds a 
large amount of heat. 


E12.10: Climate is determined by 
energy transfer from the Sun at and 
near Earth's surface. This energy 
transfer is influenced by dynamic 
processes such as cloud cover, 
atmospheric gases, and Earth’s 
rotation, as well as static conditions 
such as the positions of mountain 
ranges, oceans, seas, and lakes. 
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Exhibit 12. Earth and Space Sciences content statements for grades 4, 8, and 12 (cont.) 



Grade 4 Grade 8 Grade 12 



Earth Systems 



Biogeochemical Cycles: Uses of Earth resources (4), natural and human-induced changes in Earth 
materials and systems (8), biogeochemical cycles in Earth systems (12). 



E4.10: The supply of many Earth 
resources such as fuels, metals, fresh 
water, and farmland is limited. 
Humans have devised methods for 
extending the use of Earth resources 
through recycling, reuse, and renewal. 

E4.ll: Humans depend on their 
natural and constructed environment. 
Humans change environments in ways 
that can either be beneficial or 
detrimental for themselves and other 
organisms. 



E8.14: Water, which covers the 
majority of Earth’s surface, circulates 
through the crust, oceans, and 
atmosphere in what is known as the 
water cycle. Water evaporates from 
Earth's surface, rises and cools as it 
moves to higher elevations, condenses 
as clouds, falls as rain or snow, and 
collects in lakes, oceans, soil, and 
underground. 

E8.15: Human activities, such as 
reducing the amount of forest cover, 
increasing the amount and variety of 
chemicals released into the 
atmosphere, and intensive farming, 
have changed Earth's land, oceans, 
and atmosphere. Studies of plant and 
animal populations have shown that 
such activities can reduce the number 
and variety of wild plants and animals 
and sometimes result in the extinction 
of species. 



E12.ll: Earth is a system containing 
essentially a fixed amount of each 
stable chemical atom or element. 
Most elements can exist in several 
different chemical forms. Earth 
elements move within and between 
the lithosphere, atmosphere, 
hydrosphere, and biosphere as part of 
biogeochemical cycles. 

E12.12: Movement of matter through 
Earth’s systems is driven by Earth’s 
internal and external sources of 
energy. These movements are often 
accompanied by a change in the 
physical and chemical properties of 
the matter. Carbon, for example, 
occurs in carbonate rocks such as 
limestone, in coal and other fossil 
fuels, in the atmosphere as carbon 
dioxide gas, in water as dissolved 
carbon dioxide, and in all organisms 
as complex molecules that control the 
chemistry of life. 

E12.13: Natural ecosystems provide 
an array of basic processes that affect 
humans. These processes include 
maintenance of the quality of the 
atmosphere, generation of soils, 
control of the hydrologic cycle, 
disposal of wastes, and recycling of 
nutrients. 
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Components of Science Content as Assessment Item Contexts 



Science-literate citizens should be familiar with certain components of science content, 
such as the history and nature of science and the relationship between science and tech- 
nology. These features are highly valued by science educators and are viewed as critical 
to the teaching and learning of science (AAAS 1993; NRC 1996). In chapter three, the 
nature of science is partially addressed through a discussion of scientific inquiry. Simi- 
larly, the relationship between science and technology is partially addressed in chapter 
three through a discussion of technological design. In addition, these components of sci- 
ence content will be incorporated into the contexts of assessment items; they will not be 
directly assessed because of time and resource constraints. Chapter four contains addi- 
tional information. 

From Science Content to Science Practices 



This chapter has presented the science content that defines the NAEP Science Assess- 
ment content domain. The content statements, as presented in this chapter, do not de- 
scribe students’ performances in observable terms. Chapter three contains a description of 
science practices and cognitive demands. It also shows how science content statements 
can be combined (crossed) with science practices to generate performance expectations 
(i.e., descriptions of students’ expected and observable performances on the NAEP Sci- 
ence Assessment). Based on these performance expectations, assessment items can be 
developed and then inferences can be derived from student responses about what students 
know and can do in science. Chapter three provides an illustrative example of this 
process. 
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Chapter Three 



Science Pra c tices 

Chapter two presented content statements that define the key science principles (as well 
as the facts, concepts, laws, and theories) that NAEP will assess in 2009. However, 

NAEP will assess not only science content statements but also the ways in which knowl- 
edge is used. This chapter defines what students should be able to do with the science 
content statements by articulating key science practices that NAEP will assess: Identify- 
ing Science Principles, Using Science Principles, Using Scientific Inquiry, and Using 
Technological Design. These practices are useful for generating science-rich assessment 
items. 

To help assessment developers, the science practices can be associated with the cognitive 
demands they place on students. This chapter employs a set of four cognitive demands: 

(1) “knowing that,” (2) “knowing how,” (3) “knowing why,” and (4) “knowing when and 
where to apply knowledge.” These cognitive demands help ensure that NAEP assessment 
items are developed in a way that will elicit the kinds of knowledge and thinking that un- 
derlie the framework’s performance expectations (as explained later in this chapter). 

They also provide a tool for interpreting students’ responses on the assessment items. 

This chapter shows how science content statements can be combined or crossed with 
practices to generate performance expectations, which then guide the development of as- 
sessment items. By comparing student responses with the particular science content and 
practice being assessed, inferences about what students know (about particular science 
principles) and can do (with respect to particular science practices) are made. 

Two types of textboxes are used throughout this chapter. Clarification textboxes provide 
details on potentially confusing topics, such as the distinction between Identifying Sci- 
ence Principles and Using Science Principles. Illustrative item textboxes provide assess- 
ment items that exemplify recommendations discussed in the text. Answers to selected- 
response items are indicated within the textbox; appendix C contains scoring guides for 
constructed-response items. Although items in these textboxes may assess more than one 
content statement or practice, only the primary content and practice designations are pro- 
vided. This follows NAEP practice, which uses only primary designations for items in the 
analysis and reporting of student responses. 

Overview of Practices 



Over the course of human history, people have developed many interconnected and vali- 
dated ideas about the physical and biological world. These ideas have enabled successive 
generations to achieve an increasingly comprehensive and reliable understanding of the 
natural world. Scientific ideas are generated and verified by observing natural phenom- 
ena, finding patterns in these observations, and constructing theoretical models to explain 
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these patterns. The patterns and models can in turn be used to describe, measure, clas- 
sify, explain, and predict other observations. Science knowledge is used to reason about 
the natural world and to improve the quality of scientific thought and action. Hence, 
NAEP will assess how well 4th, 8th, and 12th grade students can engage in the following 
broadly organized science practices: 

• Identifying Science Principles 

• Using Science Principles 

• Using Scientific Inquiry 

• Using Technological Design 

Because these practices are closely related, the categories are not distinct and some over- 
lap is expected. 

The ability to communicate accurately and effectively is essential in science, and this ex- 
pectation is a strand that runs across the practices. Accurate and effective communication 
may include (but is not limited to) writing clear instructions that others can follow to 
carry out an investigation; reading and organizing data in tables and graphs; locating in- 
formation in computer databases; using audio, video, multimedia, and other technologies 
to access, process, and integrate scientific findings; using language and scientific terms 
appropriately; drawing pictures or schematics to aid in descriptions of observations; 
summarizing the results of scientific investigations; and reporting to various audiences 
about facts, explanations, investigations, and data-based alternative explanations and de- 
signs (AAAS 1993; NRC 1996; Partnership for 21st Century Skills 2004). Quantitative 
reasoning is also fundamental to science. It is the capacity not only to calculate (e.g., de- 
termine density given an object’s mass and dimensions) but also to model a system (e.g., 
determine the energy released from a chemical reaction). 

SOURCES FOR THE DEVELOPMENT OF PRACTICES 

The framework developers examined a number of sources to develop the short list of 
practices to be assessed in the NAEP Science Assessment. The most important sources 
were the Science as Inquiry sections of the National Standards and chapter 12: Habits of 
Mind in Benchmarks. The committee also consulted the National Standards and Bench- 
marks sections on Science and Technology and The Designed World, and the Validities 
of Science Inquiry Assessments project (Quellmalz et al. 2005). Conducted by SRI Inter- 
national during 2001-05, this project classified assessment items according to the inquiry 
standards discussed in the National Standards. The practices described below are found 
in most of the above sources. Cognitive research on science learning, international 
frameworks, and state standards were also used as reference points. 



7 Because natural phenomena are understood and described based on collected observations, the terms 
“phenomena” and “observations” are intricately intertwined. For ease of communication, the framework 
uses the term “observations” to represent both specific observations of a natural phenomenon and the phe- 
nomenon itself. 

NAEP 2009 Science Framework 



66 




Identifying Science Principles 



This category focuses on students’ ability to recognize, recall, define, relate, and repre- 
sent basic science principles specified in the Physical Science, Life Science, and Earth 
and Space Sciences content statements presented in chapter two. The content statements 
themselves are often closely related to one another conceptually. Moreover, the science 
principles included in the content statements can be represented in a variety of forms, 
such as words, pictures, graphs, tables, formulas, and diagrams (AAAS 1993; NRC 
1996). NAEP will assess students’ ability to describe, measure, or classify observations; 
state or recognize principles included in the content statements; connect closely related 
content statements; and relate different representations of science knowledge. The prac- 
tices assessed in this category draw on declarative knowledge (or “knowing that”), which 
is described in the Cognitive Demands section later in this chapter. Identifying Science 
Principles comprises the following general types of performance expectations: 

• Describe, measure, or classify observations (e.g., describe the position and motion 
of objects; measure temperature; classify relationships between organisms as be- 
ing predator/prey, parasite/host, producer/consumer). 

• State or recognize correct science principles (e.g., mass is conserved when sub- 
stances undergo changes of state; all organisms are composed of cells; the atmos- 
phere is a mixture of nitrogen, oxygen, and trace gases that include water vapor). 

• Demonstrate relationships among closely related science principles (e.g., connect 
statements of Newton’s three laws of motion, relate energy transfer with the water 
cycle). 

• Demonstrate relationships among different representations of principles (e.g., 
verbal, symbolic, diagrammatic) and data patterns (e.g., tables, equations, graphs). 

Identifying Science Principles is integral to all of the other science practices. 

The following two items illustrate the expectation that students will recognize correct sci- 
ence principles. The first item assesses students’ ability to correctly identify simple in- 
formation about the location of bodies within the solar system (declarative knowledge). 
More than half of the eighth graders answered it incorrectly. Thirty-five percent of the 
students thought that the Moon is sometimes closer to the Sun than to Earth. 
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Illustrative Item 

The Earth’s Moon is 



A. always much closer to the Sun than it is to the Earth. 

B. always much closer to the Earth than it is to the Sun. 

C. about the same distance from the Sun as it is from the Earth. 

D. sometimes closer to the Sun than it is to the Earth and sometimes 
closer to the Earth than it is to the Sun. 



Key: B 



E8.1, Identifying Science Principles 
Source: NAEP 2000. Grade 8. 



Illustrative Item 



Animals and plants are made up of a number of different chemical elements. 
What happens to all of these elements when animals and plants die? 

A. They die with the animal or plant. 

B. They evaporate into the atmosphere. 

C. They are recycled back into the environment. 

D. They change into different elements. 



Key: C 



L8.5, Identifying Science Principles 
Source: TIMSS 2003, Grade 8.* 



* TIMSS items appearing in the framework are copyrighted © by the International Association for the 
Evaluation of Educational Achievement. 



Using Science Principles 



Scientific knowledge is useful for making sense of the natural world. Both scientists and 
informed citizens can use patterns in observations and theoretical models to predict and 
explain observations that they make now or that they will make in the future. The prac- 
tices assessed in this category draw primarily on schematic knowledge (or “knowing 
why”) in addition to declarative knowledge, which are described in the Cognitive De- 
mands section later in this chapter. Using Science Principles comprises the following 
general types of performance expectations: 

• Explain observations of phenomena (using science principles from the content 
statements). 
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• Predict observations of phenomena (using science principles from the content 
statements, including quantitative predictions based on science principles that 
specify quantitative relationships among variables). 

• Suggest examples of observations that illustrate a science principle (e.g., identify 
examples where the net force on an object is zero; provide examples of observa- 
tions explained by the movement of tectonic plates; given partial DNA sequences 
of organisms, identify likely sequences of close relatives). 

• Propose, analyze, and/or evaluate alternative explanations or predictions. 

The following item illustrates the expectation that students will predict phenomena. 



Illustrative Item 




Look at the food web above. If the com crop failed one year, 
what would most likely happen to the robin population? 

Explain your answer. 

(See appendix C for item scoring guides.) 

L8.6, Using Science Principles 
Source: TIMSS 1999, Grade 8. 



The first two science practice categories — Identifying Science Principles and Using 
Science Principles — both require students to correctly state or recognize the science prin- 
ciples contained in the content statements. A difference between the categories is that 
Using Science Principles focuses on what makes science knowledge valuable or, in other 
words, its usefulness in making accurate predictions about phenomena and in explaining 
observations of the natural world in coherent ways (i.e., “knowing why”). Distinguishing 
between these two categories draws attention to differences in the depth and richness of 
individuals’ knowledge of the content statements. Certain actions on the part of students 
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lead to an inference of Identifying Science Principles, while other actions lead to an in- 
ference of Using Science Principles. Assuming a continuum from “just knowing the 
facts” to “using science principles,” there is considerable overlap at the boundary. The 
line between the identifying and using categories is not distinct. Consider the following 
item, which illustrates the expectation that students will connect different representations. 
In this case, the student must identify the correct pictorial representation of a complete 
electrical circuit. 




Student responses to this item are open to two interpretations. If students have had a great 
deal of exposure to these types of circuit representations, their responses would fall under 
Identifying Science Principles. However if, these circuit representations are relatively 
new for students, then they would need to apply more reasoning and their responses 
would fall under Using Science Principles. 
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The following textbox provides further illustration of the distinction between identifying 
the boiling point of water (a fact) and using the relationship between boiling point and 
pressure (altitude) to explain or predict. 



Clarification: Distinguishing Between Identifying Science Principles and Using 
Science Principles — A Boiling Point Example 

Grade 8 Content Statement: Matter — Properties of Matter: 

P8.4: . . . Each element and compound has physical and chemical properties, such as 
boiling point, density, color, and conductivity, which are independent of the amount of 
the sample. 

Distinguishing between the two categories of Identifying Science Principles and Using 
Science Principles is a function of actions or performances. Using boiling point as an 
example, one might observe different responses to the question, “What is the boiling 
point of water?” Behaviors or actions might include: 

• Penciling in the oval corresponding to 100 °C in a selected-response item. 

• Writing: “The boiling point of water is 100 °C at sea level.” 

• Writing: “The boiling point of water changes as pressure changes. So, even 
though water boils at 100 °C at sea level (1 atm pressure), it might boil at a lower 
temperature on top of a mountain because pressure is lower up there.” 

The above responses evoke different inferences about the science understanding of the 
individual responding. Both the first and second responses suggest that the question is 
only assessing knowledge of facts or the ability to identify a science principle; however, 
they illustrate the difference between recognizing a correct answer and retrieving that 
correct answer from memory. The third response contains even more sophisticated 
information, suggesting that the student can use a science principle to make predictions. 
Distinctions between these two categories can be clarified by examining student 
responses and conducting cognitive labs. 
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The following item set illustrates a combination of Identifying Science Principles and 
Using Science Principles. It also addresses the expectation that students interpret data 
presented in a graph and use the data to perform a mathematical calculation. 



Illustrative Item 

The graph below show's the distance traveled over time by a student walking 
down a hall. Use the information shown on the graph to do Numbers 7 and 8. 




0 5 10 15 20 25 30 

Time (seconds) 

7. During which time interval was the student moving the fastest? 
O A 
O B 
O C 
O D 



Key: D 



P8.14, Identifying Scientific Principles 

8. What was the average speed of the student from 0 seconds to 5 seconds? 
Average speed: 



(See appendix C for item scoring guides.) 

P8.14, Using Scientific Principles 

Source: Colorado Department of Education 2002, Grade 8. 



Using Scientific Inquiry 



Scientists make observations about the natural world, identify patterns in data, and pro- 
pose explanations to account for the patterns. Although scientists differ greatly from one 
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another in the phenomena they study and in how they go about their work, scientific in- 
quiry involves the collection of relevant data, the use of logical reasoning, and the appli- 
cation of imagination and evidence in devising hypotheses to explain patterns in data. 
Scientific inquiry is a complex and time-intensive process that is iterative rather than 
linear. Scientists are also expected to exhibit, indeed to model, the habits of mind — 
curiosity, openness to new ideas, informed skepticism — that are part of science literacy. 
This includes reading or listening critically to assertions in the media, deciding what evi- 
dence to pay attention to and what to dismiss, and distinguishing careful arguments from 
shoddy ones. These critical thinking and systems thinking skills are the basis for exercis- 
ing sound reasoning, making complex choices, and understanding the interconnections 
among systems (Partnership for 21st Century Skills 2004). Thus, using scientific inquiry 
depends on the practices described above — Identifying Science Principles and Using Sci- 
ence Principles. Moreover, in addition to involving declarative and schematic knowledge, 
Using Scientific Inquiry draws heavily on procedural knowledge (“knowing how,” e.g., 
knowing how to determine the mass of an object). This framework focuses on a few key 
inquiry practices that are practical to measure in the NAEP Science Assessment. Using 

o 

Scientific Inquiry comprises the following general types of performance expectations: 

• Design or critique aspects of scientific investigations (e.g., involvement of control 
groups, adequacy of sample). 

• Conduct scientific investigations using appropriate tools and techniques (e.g., se- 
lecting an instrument that measures the desired quantity — length, volume, weight, 
time interval, temperature — with the appropriate level of precision). 

• Identify patterns in data and/or relate patterns in data to theoretical models. 

• Use empirical evidence to validate or criticize conclusions about explanations and 
predictions (e.g., check to see that the premises of the argument are explicit, no- 
tice when the conclusions do not follow logically from the evidence presented). 

Scientific inquiry is more complex than simply making, summarizing, and explaining ob- 
servations, and it is more flexible than the rigid set of steps often referred to as the “scien- 
tific method.” The National Standards make it clear that inquiry goes beyond “science as 
a process” to include an understanding of the nature of science (p. 105) and further state 
the following: 

It is part of scientific inquiry to evaluate the results of scientific investiga- 
tions, experiments, observations, theoretical models, and the explanations 
proposed by other scientists. Evaluation includes reviewing the experi- 
mental procedures, examining the evidence, identifying faulty reasoning, 
pointing out statements that go beyond the evidence, and suggesting alter- 
native explanations for the same observations (p. 171). 



; In addition, 12th graders at the Advanced level are expected to be able to identify a scientific question for 
investigation. See appendix B for preliminary achievement level descriptions. 
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In the NAEP Science Assessment, when students use scientific inquiry they are drawing 
on their understanding about the nature of science, including the following ideas (see 
Benchmarks ): 

• Arguments are flawed when fact and opinion are intermingled, or the conclusions 
do not follow logically from the evidence presented. 

• A single example can never support the inference that something is always true, 
but sometimes a single example can support the inference that something is not 
always true. 

• If more than one variable changes at the same time in an experiment, the outcome 
of the experiment may not be clearly attributable to any one of the variables. 

• The way in which a sample is drawn affects how well it represents the population 
of interest. The larger the sample, the smaller the error in inference to the popula- 
tion. However, large samples do not necessarily guarantee representation, espe- 
cially in the absence of random sampling. 

NAEP will assess students’ abilities to use scientific inquiry in two ways: students will be 
required to do the practices specified above and students will critique examples of scien- 
tific inquiry. It is incorrect to assume that assessment of using scientific inquiry is best or 
only achieved through hands-on performance tasks and interactive computer tasks 
(ICTs). In both cases of doing and critiquing, some assessment tasks will also be pre- 
sented as paper- and-pencil items. In doing, tasks may present data tables and ask students 
which conclusions are consistent with the data. Other tasks will be presented as hands-on 
performance and/or interactive computer tasks (e.g., students collect data and present 
their results or students specify experimental conditions on computer simulations and ob- 
serve the outcomes). In regard to critiquing, students might be asked to identify flaws in a 
poorly designed investigation or suggest changes in the design to produce more reliable 
data. Tasks may be based on print or electronic media (e.g., items may ask students to 
suggest alternative interpretations of data described in a newspaper article). Chapter four 
contains more information on types of items. 
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The following middle school (grade 8) item illustrates the expectation that students will 
conduct scientific investigations. By manipulating the simulation, students gather data 
and solve the problems that are presented. 



Illustrative Item 

This interactive computer task is one module in an extended assessment of students’ abilities to 
use a range of technologies to investigate a complex problem, “Should lynx be reintroduced 
into a national park?” Students accessed, organized, and analyzed data on the number of hares 
in the park over a 25-year period, researched factors that would impact the population, and 
created a graph to analyze the trend. (See appendix C for a description of the full task.) 

This module allows students to interact with a simulated predator/prey (lynx/hare) population 
model. Students use the modeling tool to observe population trends that result from different 
parameter values for the lynx and hare populations. The screen shot below is an example of 
what students see after they have selected parameters and run the simulation. Note that it is a 
single screen shot and represents only a small subset of the many screens that students actually 
see when engaged in this interactive computer task. After students have run the modeling 
software, they are asked a series of questions (e.g., size of the hare population over time). 








L8.6, Using Scientific Inquiry 
Source: Quellmalz et al. 2004. 
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Using Technological Design 



In both the National Standards and Benchmarks, the term “technological design” 9 refers 
to the process that underlies the development of all technologies, from paper clips to 
space stations. The National Standards explain that this meaning “is not to be confused 
with ‘instructional technology,’ which provides students and teachers with exciting 
tools — such as computers — to conduct inquiry and to understand science” (p. 24). 

In the framework, using technological design describes the systematic process of apply- 
ing science knowledge and skills to solve problems in a real-world context. The National 
Standards clearly state the reason for including technological design in the science cur- 
riculum: “Although these are science education standards, the relationship between sci- 
ence and technology is so close that any presentation of science without developing an 
understanding of technology would portray an inaccurate picture of science” (p. 190). 

The National Standards define technology and its relationship to science as follows: 

As used in the Standards, the central distinguishing characteristic between 
science and technology is a difference in goal: The goal of science is to 
understand the natural world, and the goal of technology is to make modi- 
fications in the world to meet human needs. Technology as design is in- 
cluded in the Standards as parallel to science as inquiry (p. 24). 

As in scientific inquiry, the professional practice of technological design (also called en- 
gineering design) is complex and time intensive. Because NAEP addresses the subject 
area of science, the use of technological design components in the 2009 NAEP Science 
Assessment will be limited to those that reveal students’ abilities to apply science princi- 
ples in the context of technological design. Students’ abilities to Identify and Use Science 
Principles should provide the opportunities as well as the limits for assessment tasks re- 
lated to Using Technological Design. For example, if students are asked to design a 
town’s energy plan, they may be expected to consider the environmental effects of using 
natural gas versus using coal, but they would not be expected to consider the economic, 
political, or social ramifications of such a plan. 

The framework samples key components of Using Technological Design from the more 
complete descriptions found in the National Standards and Benchmarks . Using Techno- 
logical Design comprises the following general types of performance expectations, all of 
which involve students using science knowledge to accomplish the following: 

• Propose or critique solutions to problems, given criteria and scientific constraints. 

• Identify scientific tradeoffs in design decisions and choose among alternative 
solutions. 



9 This practice is discussed in some detail because it is new in NAEP Science Assessments. 
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• Apply science principles or data to anticipate effects of technological design 
decisions. 

The three components of Using Technological Design are discussed below. 

First, the technological design process is rooted in the definition of a problem that can be 
solved through a technological design process. The problem generally describes a human 
need or want and specifies criteria and constraints for an acceptable solution (Interna- 
tional Technology Education Association 2000). Only constraints that reflect the science 
content statements in this framework will be considered in developing relevant NAEP 
assessment items. The engineer who designs a bridge, for example, must take into ac- 
count the effects of wind and water currents by using relevant physics principles to simu- 
late these effects on possible structures before the bridge is built. 

Second, even if limited to the application of science principles, choosing between alterna- 
tive solutions almost always involves tradeoffs. As stated in the Benchmarks'. 

There is no perfect design. Designs that are best in one respect . . . may be 
inferior in other ways .... Usually some features must be sacrificed to get 
others. How such trade-offs are received depends upon which features are 
emphasized and which are down-played (p. 49). 

The application of science principles may be used to compare alternative technological 
solutions to see which will better solve the problem and accomplish the project’s goals. 

Finally, while the chosen solution may be intended to solve a human problem or meet a 
human need, the effects are not always as planned. When the automobile was invented, 
no one could have predicted the environmental and human health impacts of vehicle 
emissions. However, scientists and engineers must work together to apply their knowl- 
edge of the natural world to make such predictions. According to the National Standards , 
students in grades K-4 should know about the effects of design solutions: 

People continue inventing new ways of doing things, solving problems, 
and getting work done. New ideas and inventions often affect other peo- 
ple; sometimes the effects are good and sometimes they are bad. It is help- 
ful to try to determine in advance how ideas and inventions will affect 
other people (p. 140). 

In terms of cognitive demands, both declarative (knowing that) knowledge and schematic 
(knowing why) knowledge come into play for the three components of Using Techno- 
logical Design, as does strategic knowledge (knowing when and where to apply knowl- 
edge). 

The role of technological design in U.S. science classrooms currently varies widely, and 
it is not possible to predict the extent to which it will be integrated into the school 
curriculum in the future. The role of technological design in the NAEP Science 
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Assessment will need to be revisited regularly in response to its evolving role in school 
science. 

Since using technological design in the NAEP Science Assessment must have direct rele- 
vance to science, it is assumed that students have some understanding of the relationship 
between science and technology. The science-technology relationship is further discussed 
in chapter four as providing context for assessment items. 

The following item illustrates the expectation that students will apply science principles 
to anticipate the effects of a technological design decision. 
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Illustrative Item 

Occasionally, a fire will destroy a forest, burning down trees and pushing wildlife 
out of their forest homes. However, the forest will grow back. Eventually, through 
the process of forest succession as shown below, short grasses and flowers begin to 
grow and animals make new homes. 




Forest Succession 



Tall 

trees 



Short 

grasses 



Tall 

grasses 



Shrubs 



Small 

trees 



Time 



Over time, shrubs and trees begin to grow. The forest returns to a lush habitat for the 
wildlife listed in the chart below. 



Forest Wildlife 



Ground dwelling 


Worms, beetles 


Reptiles and amphibians 


American toads, wood frogs, snakes. Eastern box turtles 


Small animals 


Squirrels, chipmunks 


Medium to large animals 


Opossums, raccoons, white-tailed deer, black bears 


Airborne 


Butterflies, moths, bees, wild turkeys, red-tailed hawks, 
bald eagles 



A power company owns paid of a forest that was destroyed by a fire. The forest could 
take decades to rebuild on its own. The company’s department of environmental studies 
suggests planting new trees to help the forest rebuild. 



Using the information in the scenario: 

• Explain how planting trees could benefit the natural ecosystem. 

• Explain how planting trees could harm the natural ecosystem. 



(See appendix C for item scoring guides.) 



E8.15, Using Technological Design 
Source: Washington Assessment of Student Learning 2004, Grade 8. 
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Summary of Practices 



The general performance expectations for each of the four practices are summarized in 
exhibit 13. Dashed lines indicate that the boundaries between these categories are not dis- 
tinct, and some overlap is to be expected. 



Exhibit 13. General performance expectations for science practices 



Identifying 

Science 

Principles 


Describe, 
measure, or 
classify 
observations. 


State or 
recognize 
correct science 
principles. 


Demonstrate 
relationships 
among closely 
related science 
principles. 


Demonstrate 

relationships 

among 

different 

representations 

of principles. 








Suggest 


Propose, 




Explain 


Predict 


examples of 


analyze, and/or 


Using Science 


observations 


observations 


observations 


evaluate 


Principles 


of phenomena. 


of phenomena. 


that illustrate a 


alternative 








science 


explanations or 








principle. 


predictions. 










Use empirical 






Conduct 


Identify 


evidence to 




Design or 


scientific 


patterns in 


validate or 


Using 


critique 


investigations 


data and/or 


criticize 


Scientific 


aspects of 


using 


relate patterns 


conclusions 


Inquiry 
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appropriate 
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Using 

Technological 

Design 



Propose or critique 
solutions to 
problems given 
criteria and 
scientific 
constraints. 



Identify scientific 
tradeoffs in design 
decisions and 
choose among 
alternative solutions. 



Apply science 
principles or data to 
anticipate effects of 
technological design 
decisions. 
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Clarification: Sample Performance Expectations 
for a Life Science Content Statement 

The examples below are all related to the following grade 8 Life Science content statement: 

L8.4: Plants are producers — they use the energy from light to make sugar molecules from the 
atoms of carbon dioxide and water. Plants use these sugars along with minerals from the soil to 
form fats, proteins, and carbohydrates. These products can be used immediately, incorporated into 
the plant’s cells as the plant grows, or stored for later use. 

All examples are also related to a specific situation: 

Two different varieties of grass — one better adapted to full sunlight and one better adapted to 
shade — are each grown in sunlight and in shade. 

The results of a controlled experiment along these lines might resemble the following: 



Condition 


Grass Type A 


Grass Type B 


Sunlight 


Better growth* 


Less good growth* 


Shade 


Less good growth* 


Better growth* 



* Several variables could be used to indicate growth: mass or dry mass of plants, thickness of stems, 
number of new sprouts, etc. 



Identifying Science Principles 

1. State from where a plant’s food originates. 

2. Classify the grass plants as producers or consumers. 

The first performance calls for students to repeat information found in the content statement with 
little or no modification. The second performance asks students to use the definition of producers 
given in the content statement to classify or identify the plants. 

Using Science Principles 

1 . Predict whether sugar will move up or down the stems of the grass plants and explain 
your prediction. 

2. Explain where the mass of the growing grass originates. 

These performances require students to use principles in the content statement to predict or 
explain specific observations (growing grass in this case). The content statement itself does not 
provide the answers to the questions. 
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Using Scientific Inquiry 

1 . Given a data table showing the mass of grass plants of each type grown in the sunlight 
and shade, draw conclusions about which variety of grass is better adapted to each 
condition. 

2. List other valuables that should be controlled in order to feel confident about your 
conclusions. 

The first performance is related to the content statement in that the importance of light for plant 
growth is useful background information for students. However, the performance requires 
interpretation of new information (the data table) that has to do with differences among types of 
plants, while the content statement contains generalizations about all plants. Thus, the 
performance requires students to use the data to develop new knowledge that they did not have 
previously. The second performance is in paid an assessment of the students’ understanding of 
experimental design. However, good answers would also require knowledge of this and related 
content statements to identify valuables that arc relevant to plant growth. 

Using Technological Design 

1 . Given experimental results on the growth of different varieties of grass plants under 
sunlight and shade conditions, develop a plan for using different types of grass seed in 
different parts of a partially shaded park. 

This performance requires students to use knowledge of the content statement and the 
experimental results to accomplish a practical goal (in this case, a park with grass growing well in 
areas that receive varying amounts of sunlight). 



Performance Expectations 



The NAEP Science Assessment will focus on how students bring science content (as de- 
scribed in chapter two) to bear as they engage in the practices described in this chapter. 
That is, science practices are not content-free skills; they require knowledge of the Physi- 
cal, Life, and Earth and Space Sciences as well as knowledge about scientific inquiry and 
the nature of science (e.g., drawing conclusions from investigations). 

Performance expectations are derived from the intersection of content statements and sci- 
ence practices. If the content statements from the Physical, Life, and Earth and Space 
Sciences are the columns of a table and the practices (Identifying Science Principles, Us- 
ing Science Principles, Using Scientific Inquiry, Using Technological Design) are the 
rows, the cells of the table are inhabited by performance expectations. Examples are pro- 
vided in exhibit 14, which is based on exhibit 1 in chapter one. Note that performance 
expectation cells may overlap because the content and practice categories themselves are 
not distinct (as indicated by dashed lines). 
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Exhibit 14. Generating examples of grade 8 performance expectations 







Science Content 






Physical Science 
Content Statements 


Life Science 
Content Statements 


Earth and Space 
Sciences 

Content Statements 




Identifying 

Science 

Principles 


Identify the units that 
might be used to 
measure the speed of 
an ant and the speed of 
an airplane (see 
P8.14).* 


Identify the raw 
materials that plants 
use to make sugars 
(see L8.4). 


Identify wind as the 
movement of air from 
higher to lower 
pressure regions (see 

E8.ll). 


C/5 

<D 

%— » 
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Using Science 
Principles 


An object (e.g., a toy 
car) moves with a 
constant speed along a 
straight line. Predict 
(with justification) 
what might happen to 
this object’s speed as it 
rolls downhill (see 
P8.16). 


Explain why sugars arc 
found to move 
primarily down the 
stem of a growing 
plant (e.g., potato, 
carrot) (see L8.4). 


Explain why mountain 
soils are generally 
thinner than floodplain 
soils (see E8.6). 


5-h 

cu 

<u 

o 

c 

'o 

oo 


Using 

Scientific 

Inquiry 


Design an experiment 
to determine how the 
speed of a battery- 
operated toy car 
changes as a result of 
added mass (see 
P8.16). 


Criticize conclusions 
about likely 
consequences of 
consuming various 
diets based on flawed 
premises or flaws in 
reasoning (see L8.5). 


Given data (indexed by 
month) on annual 
trends of incoming 
solar radiation for five 
cities, determine 
whether the location is 
in the Northern or 
Southern Hemisphere 
(see E8.12). 




Using 

Technological 

Design 


Evaluate the following 
car designs to 
determine which one is 
most likely to maintain 
a constant speed as it 
goes down a hill (see 
P8.16). 


Identify possible 
ecological side effects 
of agricultural 
fertilizer runoff into a 
lake (see L8.7). 


Describe the 
consequences (e.g., 
erosion) of 
undercutting a steep 
slope for a road cut 
(see E8.4). 



* To identify the science content statement on which each performance expectation is based, the content 
statement’s unique code (from exhibits 8, 10, and 12 in chapter two) is provided. 
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The content statements from chapter two on which these performance expectations are 
based are written in general terms. The process of creating performance expectations 
requires further clarification of the content statements themselves. As described in 
chapter two, this involves “detailing” the meanings of the content statements and setting 
boundaries on the content to be assessed at a given grade level. Moreover, if the crossing 
of content statements with practices were done for every science content statement and 
practice, the number of performance expectations generated could be unmanageably 
large. Exhibit 15 provides selected example performance expectations for a single Earth 
and space sciences content statement. The specifications include additional examples. 
These examples are illustrative, not exhaustive. It is expected that assessment developers 
will continue this process for all content and practices sampled for a particular NAEP 
Science Assessment. 

Performance expectations are written with particular verbs indicating the desired per- 
formance expected of the student. The action verbs associated with each practice are not 
firmly fixed, and the use of any action verb must be contextualized. For example, when 
the science practice component “ conduct scientific investigations” is crossed with a 
states-of-matter content statement, it can generate a performance expectation that em- 
ploys a different action verb, such as “ heats as a way to evaporate liquids.” 

Generating and Interpreting Items 



Neither the content statements from chapter two nor the practices discussed in this chap- 
ter will be assessed in isolation. All assessment items will be derived from a combination 
of the two (i.e., from performance expectations). Observed student responses to these 
items can then be compared with expected student responses to make inferences about 
what students know and can do. Exhibit 15 is an Earth and Space Sciences example of 
the process of generating and interpreting items from performance expectations. The item 
examples are of two types: item suggestions (descriptions of items to be developed) and 
illustrative items (released items from various large-scale assessments). Additional 
examples of the process of generating and interpreting items are provided in the specifi- 
cations. 



NAEP 2009 Science Framework 



84 





Exhibit 15. Earth and Space Sciences example of generating and interpreting items 



Grade 8: Earth in Space and Time — Objects in the Universe 

Content Statement 

E8.2: Gravity is the force that keeps most objects in the solar system in regular and predictable motion. 
Those motions explain such phenomena as the day, the year, phases of the Moon, and eclipses. 



Commentary 

This content statement encompasses two interrelated sets of concepts: 

1. Gravity acts between and among all objects in the solar system, and it plays an essential role in the 
regular and predictable motions of planets around the Sun and of satellites around planets. 

• On Earth, gravity is experienced as a force that pulls everything “down” toward the center of the 
Earth. (A common naive conception is that the atmosphere “pushes” things down and causes 
gravity.) 

• Gravity is a force of attraction that is exerted by every object on every other object. 

• Gravity exists in space and on other planets. (A common naive conception among students is that 
there is no gravity in space because space has no air.) 

• The almost circular motion of planets and satellites results from the force of gravity and the 
tendency of a body to continue moving through space in a straight line unless acted upon by a net 
force. 

2. The regular and predictable motions of Earth, the Sun, and the Moon cause the cyclic phenomena that 
can be observed in the sky. 

• The day-night cycle results from Earth’s rotation on its axis once in 24 hours. 

• Annual changes in the visible constellations and the seasons result from Earth's revolution around 
the Sun once every 365% days. 

• Moon phases result from the Moon's orbit around Earth about once a month, which changes the 
part of the Moon that is lighted by the Sun and how much of the lighted part can be seen from 
Earth. 

Note the connection between this content statement and the Physical Science subtopic, forces 
affecting motion. 

Students are not expected to use the inverse square relationship of gravitational force and distance to find 
the strength of the gravitational force between two objects. 

Students do not need to know that the motion of planets and satellites is elliptical and not circular. 



Examples of Performance Expectations 

Identifying Science Principles. Students can: 

• Identify gravity as the force exerted by every object in the solar system on every other object. 

• Identify gravity as the force that keeps the Moon circling Earth, rather than flying off into space. 

• Describe the regular motions of Earth through space, including its daily rotation on its axis and its 
yearly motion around the Sun. 
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Exhibit 15. Earth and Space Sciences example of generating and interpreting items (cont.) 



Examples of Performance Expectations (cont.) 

Using Science Principles. Students can: 

• Explain that the orbit of one object around another is due to the tendency of an object to move in a 
straight line through space and due to the force of gravity between the two objects. 

• Explain how the monthly pattern of Moon phases observed from a point on Earth results from the 
Moon’s orbit around Earth, which changes the part of the Moon that is lighted by the Sun and 
what portion of the lighted part can be seen from Earth. 

• Distinguish between explanations for lunar (Moon) phases and lunar eclipses. 

• Explain that astronauts and other objects in orbit seem to “float” because they are in free fall, 
under the influence of gravity. 

Using Scientific Inquiry. Students can: 

• Arrange a set of photographs of the Moon taken over a month's time in chronological order and 
explain the order in terms of a model of the Earth-Sun-Moon system. 

• Design a plan for observing the Sun over a year’s time to find out how the length of the day is 
related to the rising and setting point of the Sun on the horizon. 

• Design a series of observations or measurements to determine why some objects — such as certain 
asteroids or comets — visit the solar system just once, never to return. 

Using Technological Design. Students can: 

• Choose among several (qualitative) methods for aiming a rocket so that it reaches the planet Mars 
and give a rationale that shows understanding of orbital motion. 

• Use scientific tradeoffs in deciding whether or not to support a plan to observe and predict orbits 
of asteroids that enter the inner solar system. 

• Given a scenario in which a person is shipwrecked on an island in the ocean, critique plans to 
create a calendar to keep track of the passage of time. 



Items To Assess Identifying Science Principles 

Illustrative Item 1 

What force keeps the planets in our solar system in orbit around the Sun? 



A. 


gravitational 


B. 


magnetic 


C. 


electrical 


D. 


nuclear 


Key: A 





Source: Adapted from Massachusetts Department of Education, Massachusetts Comprehensive Assessment 

System (MCAS) 2000, Grade 8.* 



* MCAS materials appearing in the framework have been released to the public by the Massachusetts 
Department of Education and are available at no cost at http://www.doe.mass.edu/mcas/testitems.html. 
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Exhibit 15. Earth and Space Sciences example of generating and interpreting items (cont.) 



Items To Assess Identifying Science Principles (cont.) 

Illustrative Item 2 



The drawings show a rocket being launched from Earth and returning. 





In which of these positions does gravity act on the rocket? 



A. 


position 3 only 




B. 


positions 1 and 2 only 




C. 


positions 2 and 3 only 




D. 


positions 1, 2, and 3 




Key: D 




Source: TIMSS 1999, Grade 8. 



Items To Assess Using Science Principles 



Illustrative Item 



A space station is to be located between the Earth and the Moon at the place where 
the Earth’s gravitational pull is equal to the Moon's gravitational pull. 

On the diagram below, circle the letter indicating the approximate location of the 
space station. 




Explain your answer. 



Source: NAEP 1996, Grade 8. 



NAEP 2009 Science Framework 



87 





Exhibit 15. Earth and Space Sciences example of generating and interpreting items (cont.) 



Items To Assess Using Science Principles (cont.) 

Interpretation: The correct answer is C. Since the Moon has one-sixth the amount of gravity as Earth, a 
body that experiences an equal gravitational force from Earth and the Moon should be closer to the Moon. 
Point C is the only point that is closer to the Moon. Note: Point C is about one-twelfth of the way between 
the Moon and Earth; it should be one-sixth of the distance. (See appendix C for the item scoring guide.) 

Item Suggestion 

Is there gravity in space? Which of the following gives the best response to this question? 

A. No. You can see that astronauts float around weightless in their cabin. 

B. No. There is no air in space, so gravity cannot exist there. 

C. Yes. There must be gravity since planets keep circling the Sun. 

D. Some. The Moon has one-sixth as much gravity as Earth, so we know there is some gravity in 
space. 

Key: C 

Interpretation: The correct answer is C. This question is drawn from a series of studies that show a common 
naive conception — that there is no gravity in space because space has no air. The distractors are drawn 
from student interviews. It is likely that these naive conceptions stem from images that students have seen 
of astronauts floating around in a “weightless” environment while in orbit. This item probes schematic 
knowledge (see Cognitive Demands section later in this chapter). 



Item To Assess Using Scientific Inquiry 

Item Suggestion 

A student is presented with a set of photographs of the Moon taken over a month's time. The photos 
are not presented in chronological order. The student is asked to arrange them in the order in which 
they were taken and explain the reason for Moon phases. 

Interpretation: This suggestion reflects items used frequently in curricular materials (e.g., Schatz and 
Cooper 1994). Students are asked to find patterns in the data. First, they should be sufficiently familiar with 
the lunar cycle to arrange the photographs in order, either in a line to represent a chronology or in a circle 
to represent a cycle (tapping declarative knowledge and procedural knowledge to a lesser extent — see 
Cognitive Demands section later in this chapter). Then, students should be able to explain Moon phases in 
terms of the Moon circling Earth and the changing angle between the Sun and Moon as observed from 
Earth. This part of the item probes schematic knowledge — see Cognitive Demands section later in this 
chapter. This is a challenging question that many educated adults fail to answer. However, studies show 
that middle school students can learn to do this exercise by observing lunar phases and explaining them 
using a model of the Earth-Sun-Moon system (Barnett and Morran 2002; Kavanagh, Agan, and Sneider 
2005). 
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Exhibit 15. Earth and Space Sciences example of generating and interpreting items (cont.) 



Items To Assess Using Technological Design 

Item Suggestion 1 

NASA wants to launch a spacecraft with rockets from Earth so that it will reach and orbit Mars. Which 
of the following statements about this flight is WRONG? 

A. In the first phase of its flight, the forces acting on the spacecraft are the thrust of the rocket engine, 
gravity, and friction from the Earth's atmosphere. 

B. When the rocket engine shuts off, the only force acting on the spacecraft is the force of gravity. 

C. Once the spacecraft is above the Earth’s atmosphere and the rocket engine is off, it will travel at a 
constant speed since there is no gravity in space. 

D. If the spacecraft is aimed correctly and has the proper speed, the spacecraft will reach Mars and 
require only engine braking to attain orbit. 

Key: C 

Interpretation: The correct answer is C because there is gravity in space and planning for such a rocket 
flight would need to take into account the gravity from Earth, Mars, and the Sun (declarative knowledge — 
see Cognitive Demands section later in this chapter). This question is drawn from a series of studies that 
show the following naive conceptions about gravity are common among many students at the middle 
school, high school, and even college levels: If a body is moving, there is a force acting on it in the 
direction of motion (Finegold and Gorsky 1991; Gunstone and Watts 1985; Sequeira and Leite 1991); there 
is no gravity in space (Bar et al. 1994; Chandler 1991; Morrison 1999); and gravity cannot act in space 
because there is no air in space (Bar and Zinn 1998). One study showed that, with effective instruction, 
middle school students can overcome these naive conceptions and learn that gravity does, in fact, act in 
space, where it keeps satellites and planets in their orbits (Bar, Sneider, and Martimbeau 1997). 

Item Suggestion 2 

Decisions about whether or not to develop new technologies always concern tradeoffs. For example, 
most scientists today believe that the extinction of dinosaurs and many other species was caused by the 
collision of a large asteroid with Earth 65 million years ago. As a result, there is a proposal to develop 
two new technologies: (1) the detection and tracking of all asteroids large enough to do considerable 
damage if they should strike Earth, and (2) the development of a means to send a spacecraft to meet the 
asteroid in space and change its path. Write a paragraph describing: 

a. whether or not you think it would be possible to develop these technologies based on your 
knowledge of science; and 

b. some of the scientific tradeoffs that should be considered in deciding whether or not to develop 
these new technologies. 

Interpretation: Look for evidence that the students understand what asteroids are, that scientists have 
observed asteroids, and that observations taken at several points in time allow for the prediction of an 
asteroid's path. Also look for evidence that students understand that spacecraft can be built, launched, and 
navigated to intercept solar system bodies. (In fact, several spacecraft have already intercepted asteroids 
and comets.) Regarding scientific tradeoffs, look for evidence that students recognize the advantages of the 
proposed technologies (e.g., avoid a catastrophic collision in which billions of people and animals could 
die) as well as possible negative effects (e.g., breakup of the asteroid so there are many collisions rather 
than one, accidents on launch). 
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Learning Progressions 



A learning progression is a sequence of successively more complex ways of reasoning 
about a set of ideas. For any important set of ideas in science, understanding increases 
over time as students learn more, moving from initially naive knowledge of the natural 
world to increasingly more sophisticated knowledge and conceptual understanding; this 
typically occurs in conjunction with educational experiences in and out of school (NRC 
2001). In other words, the progression from novice learner to competent learner to expert 
begins with the acquisition of relevant experiences, principles, concepts, facts, and skills 
and moves to the accumulation and organization of knowledge in a specific domain and 
finally to expertise after extensive experience and practice (e.g., Ericsson 2002). The 
attention paid to growth of understanding may yield rich information about student 
progress. 

Research has been conducted on students’ learning progressions in some areas of science 
and at some levels of students’ development. It is expected that this research will directly 
inform the development of assessment items at grades 4, 8, and 12. For example, NRC 
has commissioned papers on learning progressions in evolution (Catley, Lehrer, and 
Reiser 2005) and in atomic molecular theory (Smith et al. 2004). Learning progressions 
provide opportunities for assessing specific content in greater depth. 

Several caveats about learning progressions are in order. First, learning progressions are 
not developmentally inevitable but depend on instruction interacting with the student’s 
prior knowledge and construction of new knowledge. Thus, learning progressions need to 
invoke assumptions about instruction. Second, there is no single “correct order”; there 
may be multiple pathways by which certain understandings can be reached. The particu- 
lar pathway taken may be influenced by prior instructional experiences, individual differ- 
ences, and current instruction (NRC 1999c, 2001). Thus, learning progressions will nec- 
essarily be complex, involving multiple specific paths at the micro level, and will need to 
be described in ways that encompass such diversity. Third, actual learning is more like 
ecological succession with changes taking place simultaneously in multiple intercon- 
nected ways. Thus, attempts to describe specific sequences of learning performance (in- 
cluding those in the Catley, Lehrer, and Reiser (2005) and Smith et al. (2004) papers) 
must inevitably be artificially constrained and ordered. Finally, the learning progressions 
suggested in the framework and specifications are partly hypothetical or inferential be- 
cause long-term longitudinal accounts of learning by individual students do not exist. 

Exhibit 16 uses a set of related science content statements across grades 4, 8, and 12 and 
follows the format of the science practices defined in the framework. It illustrates how 
relevant research (e.g., Smith et al. 2004) can be used as an opportunity to assess content 
in greater depth — available research on student learning is used to inform the generation 
of related performance expectations across grades. The exhibit includes examples of per- 
formance expectations for a possible learning progression for states of matter. These il- 
lustrative performance expectations do not denote a sense of content priority or impor- 
tance and are not a complete representation of the research currently available. 
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Exhibit 16. Examples of performance expectations for states of matter 



Grade 4 

(See content statement P4.3.) 


Grade 8 

(See content statement P8. 1.) 


Grade 12 

(See content statement P12.1.) 


Identifying Science Principles 

Classify samples of material as 
solid, liquid, or gas. 


Identifying Science Principles 

Given an animation of molecules 
in motion, identify the substance 
that is being illustrated as a solid, 
liquid, or gas. 


Identifying Science Principles 

Explain why ice is harder than 
liquid water in terms of the 
strength of the force between the 
molecules. 


Using Science Principles 

Infer that a change of state (e.g., 
freezing or melting ) affects the 
identity of an object but not the 
identity of the material of which 
it is made. 


Using Science Principles 

Predict how the mass of a sample 
of iodine will change after 
sublimation. Justify the prediction 
based on what occurs during 
sublimation at a molecular level. 


Using Science Principles 

Use the concept of molecular 
arrangements and bonds to 
explain why graphite is very soft 
and diamond is very hard, even 
though they are both made of 
pure carbon. 


Using Scientific Inquiry 

Collect, display, and interpret 
data showing how the 
temperature of a substance 
changes over time as it cools and 
becomes a solid. 


Using Scientific Inquiry 

Plan and conduct an investigation 
to determine the melting point 
and boiling point of an unknown 
substance. 


Using Scientific Inquiry 

Using molecular theory, explain 
the results of experiments 
showing how the volume of three 
different liquids changes when 
they are heated. 


Using Technological Design 

Propose a method for determining 
for certain whether holiday 
chocolates that have been shaped 
by different processes (melting, 
freezing, reshaping, or breaking 
into pieces) have the same 
amount of chocolate in them. 


Using Technological Design 

Choose the best solution for 
increasing the altitude of a hot air 
balloon based on an 
understanding of the macroscopic 
and microscopic changes that 
occur when the gas inside the 
balloon is heated. 


Using Technological Design 

Design an instrument to measure 
temperature as accurately as 
possible, taking into account both 
the thermal properties of liquids 
and solids to be used in the 
device and the structural shape 
and dimensions of the device. 



Cognitive Demands 



The four science practices (Identifying Science Principles, Using Science Principles, Us- 
ing Scientific Inquiry, and Using Technological Design) articulate what students should 
know and be able to do with the science principles presented in chapter two. Certain ways 
of knowing and reasoning (cognitive demands) underpin these four science practices. In 
this section, the four cognitive demands — “knowing that,” “knowing how,” “knowing 
why,” and “knowing when and where to apply knowledge” — are discussed briefly (see 
the specifications for more information). The goal is to further clarify the descriptions of 
the science practices, to facilitate item specifications and item writing, and to provide a 
framework for interpreting students’ responses. That is, the set of four cognitive demands 
can be used as a lens to facilitate item development and to analyze student responses (Li 
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and Shavelson 2001; Shavelson 2006; Shavelson, Ruiz-Primo, and Wiley 2005), thereby 
checking expectations regarding the science content and practice(s) that are being tapped 
by a given assessment item. 10 

“Knowing that” refers to declarative knowledge. This cognitive demand sets up the ex- 
pectation that students should know and reason with basic science facts, concepts, and 
principles (e.g., density is mass per unit volume) and that they should be able to recall, 
define, represent, use, and relate these basic principles as appropriate. This cognitive de- 
mand corresponds most closely to the science practice of Identifying Science Principles. 

“Knowing how” refers to procedural knowledge. This cognitive demand sets up the ex- 
pectation that students can apply the science facts, concepts, and principles in doing sci- 
ence. For example, students should know how to perform simple (routine) and complex 
procedures such as systematically observing and recording which objects sink and float in 
water, using a balance scale, measuring an object’s mass, calculating an object’s density, 
and designing and interpreting the results of an investigation (e.g., manipulating one vari- 
able and holding others constant). Procedural knowledge underlies much of the science 
practice of Using Scientific Inquiry as defined in this framework. 

“Knowing why” refers to schematic knowledge. This cognitive demand sets up the ex- 
pectation that students can explain and predict natural phenomena as well as account for 
how and why scientific claims are evaluated, argued and justified, or warranted (explain- 
ing and reasoning with principles and models). That is, this cognitive demand deals with 
students’ understanding of how the natural world works (such as why some things sink 
and others float in water), why light is essential to the propagation of most plants, or why 
the Moon changes phases. This cognitive demand overlaps considerably with the science 
understanding expected in the practice of Using Science Principles and also in the prac- 
tices of Using Scientific Inquiry and Using Technological Design. 

The last cognitive demand, “knowing when and where to apply knowledge,” or strategic 
knowledge, is commonly talked about as transfer of current knowledge to new situations 
(tasks or problems). Strategic knowledge involves knowing when and where to use sci- 
ence knowledge in a new situation and reasoning through the novel task to reach a goal. 
Strategic knowledge sets up the expectation that students can take their current knowl- 
edge and apply it to a somewhat novel situation. Such adaptation of knowledge to a par- 
ticular problem and context underlies especially the practices of Using Scientific Inquiry 
and Using Technological Design. 

The cognitive demands are related, not independent (similar to the science practices). 

That is, when explaining “why,” a student will need to call on “knowing that”; at times, 
in justifying “why,” a student may have to call on “knowing how.” In addition, depend- 
ing on the novelty of the task, strategic knowledge (knowing when and where to apply 



10 More than one cognitive demand can be associated with the more complex science practices. These asso- 
ciations may shift according to the knowledge that students at different grade levels bring to an assessment 
task. 
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knowledge) may be called into play. Nevertheless, these related cognitive demands can 
be distinguished, and it is helpful to do so for item development and interpretation of 
student responses. 
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Chapter Four 



Overview of the Assessment Design 

This chapter provides an overview of the specifications. It begins with a brief description 
of the 2009 NAEP Science Assessment and a discussion of how items can be set in cer- 
tain contexts (e.g., history and nature of science) to illustrate components of science con- 
tent that are not otherwise incorporated in the content statements. The types of items to be 
included in the assessment are described and examples are provided in illustrative item 
textboxes. Answers to selected-response items are indicated in the textbox; scoring 
guides for constructed-response items are provided in appendix C. Although items in the 
textboxes may assess more than one content statement or practice, only the primary con- 
tent and practice designations are provided. This follows NAEP practice, which uses only 
primary designations for items in the analysis and reporting of student responses. To cap- 
ture the wide range of science content statements and practices, the assessment will con- 
tain an array of item types. Consideration is given to English language learners and stu- 
dents with disabilities. The chapter concludes with recommendations for small-scale 
special studies. 

Overview of the Science Assessment 



The NAEP Science Assessment will include items sampled from the domain of science 
achievement identified by the intersection of the content areas and science practices (i.e., 
performance expectations) at grades 4, 8, and 12. The assessment will include selected- 
response (multiple-choice) items and constructed-response items (which include short 
and extended constructed-response items as well as concept-mapping tasks). Some com- 
bination items may require more than one response; they include item clusters, POE item 
sets, hands-on performance tasks, and interactive computer tasks. The responses re- 
quested may be all selected-response, all constructed-response, or a mixture. At each of 
grades 4, 8, and 12, student assessment time will be divided evenly between selected- 
response items and constructed-response items. Extra assessment time will be provided 
for a portion of the student sample so that hands-on performance tasks and interactive 
computer tasks can be administered. 

At grade 4, the items will be distributed approximately evenly among Physical Science, 
Life Science, and Earth and Space Sciences. At grade 8, there is a somewhat greater em- 
phasis on Earth and Space Sciences. At grade 12, the balance shifts toward Physical Sci- 
ence and Life Science, with less emphasis on Earth and Space Sciences. 

Finally, the distribution of items across the science practices will be approximately 60 
percent combining Identifying Science Principles and Using Science Principles, 30 per- 
cent Using Scientific Inquiry, and 10 percent Using Technological Design. From grade 4 
to 8 to 12, the emphasis on Using Science Principles increases and the emphasis on Iden- 
tifying Science Principles decreases. As students progress through the grades, it is 
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expected that their critical response skills and methodological and analytical capabilities 
will increase. 

Assessment Item Contexts 



Science-literate citizens should be familiar with certain components of science content, 
such as the history and nature of science and the relationship between science and tech- 
nology. Chapter three addresses the nature of science largely through a discussion of the 
science practices (particularly Using Science Principles and Using Scientific Inquiry). 

The relationship between science and technology is partially addressed in a discussion of 
the Using Technological Design practice. The history and nature of science not only clar- 
ify facets of science practices but also the human aspect of science and the role science 
has played in various cultures. Students can see that science changes, and new conclu- 
sions can be reached based on new empirical data (e.g., development of the theory of 
plate tectonics, use of gaps in early versions of the Periodic Table to “discover” new ele- 
ments). The reciprocal relationship between science and technology can be seen, for ex- 
ample, in that scientists use technological tools to empirically test proposed explanations 
for questions about the natural world and engineers develop adaptations to the natural 
world to address human problems, needs, and aspirations based in part on science. 

When items are written to particular content statements, they may be framed in these con- 
textual components of science content. Aspects of the history and nature of science and 
the relationship between science and technology should thus be incorporated into the con- 
texts of assessment items as shown below. 



Illustrative Item 

Ernest Rutherford found that when he fired alpha particles at a thin gold foil, some 
were scattered at large angles.* What caused this scattering? 

A. The gold’s positive atomic nuclei attracted the negatively charged alpha particles. 

B. The gold’s negative atomic nuclei repelled the negatively charged alpha particles. 

C. The gold’s negative atomic nuclei attracted the positively charged alpha particles. 

D. The gold’s positive atomic nuclei repelled the positively charged alpha particles. 



Key: D 



PI 2.2, Identifying Science Principles 



* The framework's developers created this item for illustrative purposes; it requires further development in 
that it is not field tested or published. 
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Types of Items 



The judicious selection of items lies at the heart of any effective assessment of science 
achievement. The Framework for the 1996-2005 NAEP Science Assessments called for 
multiple-choice items (selected response), open-ended paper-and-pencil items (con- 
structed response), and performance exercises. Multiple-choice items made up about 40 
percent of the assessment, as measured by student response time, and open-response 
items made up about 60 percent of assessment time. In addition, subsets of the students 
sampled were given an extra 20 minutes in grade 4 and 30 minutes in grades 8 and 12 to 
complete hands-on performance tasks. This framework generally follows the 1996-2005 
recommendations in item structure but also specifies additional item types — some se- 
lected response and others constructed response. As noted above, the 2009 recommenda- 
tion for item distribution by item format, in terms of student response time, is 50 percent 
selected response and 50 percent constructed response. 

Two additional considerations are the need to develop items that probe students’ ability 
to use communication skills and quantitative reasoning skills in science (see p. 64). 
Although this framework does not prescribe the amount of assessment time to be spent on 
items that require specific forms of communication or application of mathematics, it is 
expected that items requiring these skills will be represented at all three grade levels. 
Further details will be provided in the specifications. 

JUSTIFICA TIONFOR VARIA TIONIN TYPES OF ITEMS 

Issues of time and cost are paramount in any assessment. Accordingly, most of the item 
formats on the NAEP Science Assessment will be traditional selected response and short 
constructed response. However, some more complex items (e.g., hands-on performance 
tasks) should be part of any science assessment. 

Responses to complex items often correlate positively and cluster with responses to sim- 
pler items. However, complex items are recommended in the framework for the follow- 
ing reasons: 

• Items may correlate positively with one another, but they do not necessarily 
measure the same thing; that is, positive correlations can arise even when the cog- 
nitive demands of the assessment items vary. Research has shown that items vary 
in their cognitive demands for different kinds of knowledge and reasoning (e.g., 
Leighton 2004). 

• The NAEP Science Assessment signals the kinds of responses to tasks, problems, 
and exercises, along with the kinds of knowledge and reasoning, that should be 
expected of students as a result of what is taught in the science curriculum (con- 
sistent with the National Standards and Benchmarks). 
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For these reasons, the framework calls for a variety of item types for the 2009 NAEP Sci- 
ence Assessment (see the specifications for further details and explanation). 



DEFINITIONS OF TYPES OF ITEMS 

The framework distinguishes selected-response from constructed-response item formats. 
For selected-response formats, students respond to a question by selecting the answer 
they believe to be most scientifically justifiable from a given set of alternatives. In con- 
trast, with constructed-response formats, students respond to a question by “generating” 
or “constructing” a response. The constructed-response might be a single word, a short 
answer, an essay explanation, a summary of a laboratory investigation using concrete ma- 
terials, or typed responses to a computer simulation. 

In addition to these two main item formats, there are combination items that generally 
require more than one response. These include item clusters, POE item sets, hands-on 
performance tasks, and interactive computer tasks. These combination items can use an 
all-selected-response format, an all-constructed-response format, or a mixture of these 
two main item formats. 

Following are the main types of items to be used on the 2009 NAEP Science Assessment: 

1 . Selected response 

— Individual multiple-choice items 

2. Constructed response 

— Short constructed-response items 
— Extended constructed-response items 
— Concept-mapping tasks 

3. Combination 
— Item clusters 
— POE item sets 

— Hands-on performance tasks 
— Interactive computer tasks 

Item clusters and POE item sets may use selected-response items, constructed-response 
items, or both. For example, a set of POE items might include a multiple-choice item in 
which students select a prediction and a short constructed-response item in which stu- 
dents are asked to write a justification for their prediction. 

Hands-on performance tasks and interactive computer tasks also may use selected- 
response items, constructed-response items, or both. In recording their answers, students 
may be asked to respond to both selected-response and constructed-response items. For 
example, 12th graders might be asked to manipulate a computer simulation of a chemical 
reaction, a multiple-choice question could ask students to choose the correct mass of a 
reaction product, and a short constructed-response question could ask students to describe 
how mass is conserved in chemical reactions. 
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SELECTED-RESPONSE ITEMS 



Selected-response items include individual multiple-choice items. These types of items 
are described below. 

Individual Multiple- Choice Items 

Selected-response items most often are in a multiple-choice format. Students read, reflect, 
and then select an answer from, for example, four alternatives provided. The alternatives 
include the most scientifically justifiable response (the answer) as well as three “distrac- 
tors.” The distractors should appear plausible to students but should not be scientifically 
justifiable; when feasible, the distractors should also draw from current understanding 
about students’ mental models and learning progressions. Whenever possible and espe- 
cially when the focus is on using science principles or “knowing why” (schematic knowl- 
edge), naive conceptions, explanations, and predictions of the natural or manmade world 
should serve as distractors. 

Of the following two multiple-choice items, both require identifying science principles 
and both tap the cognitive demand “knowing that” (declarative knowledge). The first 
item taps simple factual content. The second item taps more conceptually sophisticated 
content. For some students, if they cannot easily recall the science content, the second 
item may require the practice of using science principles (i.e., tapping more of “knowing 
why” than “knowing that”). 



Illustrative Item 



Air is made up of many gases. Which gas is found in the greatest amount? 

A. Nitrogen 

B. Oxygen 

C. Carbon dioxide 

D. Hydrogen 



Key: A 



E8.7, Identifying Science Principles 
Source: TIMSS 1995. Grade 8. 
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Illustrative Item 




60 N 
30 N 
Equator 
30 S 
60S 



The diagram above shows a map of the world with the lines of 
latitude marked. Which of the following places marked on the map is 
most likely to have an average yearly temperature similar to location ©? 



A © 
B 0 
C 0 

D. © 

Key: A 



E8.12, Identifying Science Principles 
Source: TIMSS 2003, Grade 8. 



CONSTRUCTED-RESPONSE ITEMS 

Constructed-response items include short constructed-response items, extended con- 
structed-response items, and concept-mapping tasks. 

Short Constructed-Response Items 

This item type generally requires students to supply the correct word, phrase, or quantita- 
tive relationship in response to the question provided in the item; illustrate with a brief 
example; or write a concise explanation for a given situation or result. Thus, students 
must generate the relevant information rather than simply recognize the correct answer 
from a set of given alternatives, as in selected-response items. Following is an example of 
a short constructed-response item. 
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Illustrative Item 

On a hot, humid day the air contains a lot of water vapor. What happens to the water 
vapor in the air when the air becomes very cold? 

(See appendix C for item scoring guides.) 

P4.6, Identifying Science Principles 

Source: TIMSS 2003, Grade 4. 



Extended Constructed-Response Items 

This item type is generally multidimensional; that is, it taps into multiple content state- 
ments, practices, and/or cognitive demands. These types of items can provide useful 
insight into students’ levels of conceptual understanding and reasoning. They can also be 
used to probe students’ ability to communicate in the sciences. Such items generally 
present a situation within or across content areas and require students to analyze the 
situation, choose and carry out an alternative plan for addressing it, and interpret their 
response in light of the original situation. Students may also be given an opportunity to 
explain their responses, reasoning processes, or approach to a problem situation. How- 
ever, care must be taken (especially with 4th graders and English language learners) to 
ensure that language ability is not confused with science ability. 

The following item involves reasoning with mental models (on carbon cycling) and thus 
attempts to probe the practice of Using Science Principles and taps into the cognitive de- 
mand of “knowing why.” 



Illustrative Item 



The biosphere (living organisms), the lithosphere (rocks and soils of Earth’s crust), and 
the atmosphere are all involved in the cycling of carbon atoms. Describe the role that 
each plays in the carbon cycle. 



(See appendix C for item scoring guides.) 



E12.12, Using Science Principles 
Source: New Standards Spring Field Test 1999 for High School.* 



* The New Standards Spring Field Test 1999 for High School is the property of the University of 
Pittsburgh and the National Council on Education and the Economy (NCEE) and may not be used, 
reproduced, or distributed without the express written permission of the University of Pittsburgh and 



NCEE. 
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Concept-Mapping Tasks 

Concept-mapping tasks may be considered a complex item type because of the cognitive 
demands placed on students. Concept maps can be used as a reliable and valid assessment 
of students’ ability to make connections among science principles (Ruiz-Primo and 
Shavelson 1996a). Thus, concept-mapping tasks tap a science ability that is difficult to 
measure by other means. These tasks address the practice of Identifying Science Princi- 
ples and the cognitive demand of “declarative knowledge,” in particular the organization 
of this knowledge. 

In a concept-mapping task, students should be given a set of six to eight concept terms 
and be asked to construct a map linking pairs of terms with directed arrows. Students 
should label each arrow with a word or phrase that explains the relationship between a 
pair of concept terms. An arrow-linked pair of concept terms is called a proposition. Stu- 
dents’ concept maps can be evaluated as to the accuracy of propositions in their maps. 

The following textbox provides an illustrative set of instructions for a concept-mapping 
task and a sample student response. See the specifications for information about scoring 
concept maps. 
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Illustrative Item 

Once students are familiar with how to construct concept maps, they might encounter a set of task 
instructions that resemble the following (adapted from Ruiz-Primo et al. 2001, p. 107): 

Examine the concept terms listed below. The terms selected focus on the topic [insert topic]. 

Construct a concept map using the terms provided below. Organize the terms in relation to one 
another in any way you want. Draw an arrow between the terms you think are related. Label the 
arrow using phrases or only one or two linking words. 

You can construct your map on the blank pages attached. When you finish your map, check that 
(1) all the lines have an arrow, (2) all the arrows have labels, (3) your concept map uses all the 
terms provided, and (4) your map shows what you know about [insert topic]. 

List of terms: [insert list of terms] 

An 1 1 -year-old student constructed the following concept map using terms associated with the 
water cycle: 





E8.ll, Identifying Science Principles 
Source: White and Gunstone 1992, p. 16. 
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COMBINA TION ITEMS 



Combination items include item clusters, POE item sets, hands-on performance tasks, and 
interactive computer tasks. Combination items may consist of an all selected-response 
format, an all constructed-response format, or a mixture of both formats. 

Item Clusters 

The NAEP Science Assessment should include item clusters, and their development 
should be guided by current research on different forms of these items. In this type of 
item set, two or more items focus on an important idea or “mental model.” Hence, these 
items tap the practice of Using Science Principles and the cognitive demand of “knowing 
why.” Where a rigorous body of research is available on students’ conceptions (as there 
is about the solar system), item clusters provide opportunities to assess students’ under- 
standing of a particular key science principle at some depth. This type of item set can 
probe the conceptions and mental models that underlie students’ explanations of and rea- 
soning about the natural world. For example, the following questions were part of a set of 
cluster selected-response items probing high school students’ mental models in astron- 
omy (percentages of student responses to each option are given in parentheses): 



Illustrative Items 



What causes day and night? 

A. The earth spins on its axis. (66%) 

B. The earth moves around the Sun. (26%) 

C. Clouds block out the Sun’s light. (0%) 

D. The earth moves into and out of the Sun’s shadow. (3%) 

E. The Sun goes around the earth. (4%) 



Key: A 



E8.2, Using Science Principles 



The main reason for its being hotter in summer than in winter is: 



A. The earth’s distance from the Sun changes. (45%) 

B. The Sun is higher in the sky. (12%) 

C. The distance between the northern hemisphere and the Sun changes. (36%) 

D. Ocean currents carry warm water north. (3%) 

E. An increase occurs in “greenhouse” gases. (3%) 



Key: B 



E8.12, Using Science Principles 
Source: Sadler 1998, pp. 274-276. 
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Another approach to probing students’ conceptions of the natural world is to develop a 
cluster of ordered multiple-choice items. These items track students’ performance along a 
learning progression from naive understanding through more reasoned naive conceptions 
to full and scientifically justified understanding. In this approach, the progression is first 
described and then divided into levels so that multiple-choice items can be designed spe- 
cifically to assess the performance level that a student (or group of students) has reached. 
As noted earlier, constructed-response formats are not precluded for use in item clusters. 

POE Item Sets 

POE item sets ask the student to predict, observe, and/or explain as follows: A situation is 
described; the student’s task is to provide a prediction for what will happen (sometimes 
with justification) and/or to provide an explanation for what appears to be an anomaly. 
POE items tend to tap the practice of Using Science Principles and the cognitive demand 
of “knowing why” (schematic knowledge). For example, the following POE item was 
used with middle school students and focuses on prediction based on a mental model of 
buoyancy: 



Illustrative Item 

Rich cut block A into two unequal parts.* Part B is 2/3 of the original block A, and part C 
is 1/3 of the original block A. Block A sinks in water. What will happen to B and C when 
placed in water? 



A 



*• 



B 



C 



A. Both B and C will float. 

B. B will sink; C will float. 

C. B will subsurface float; C will float. 

D. Both B and C will sink. 



Key: D 



P8.4, Using Science Principles 
Source: Adapted from Shavelson 2006, and Shavelson, Ruiz-Primo, and Wiley 2005. 



* This item assumes that the entire block consists of a completely homogeneous material. The possibility 
that the block is made of heterogeneous material is unlikely to occur to middle school students. 



As noted earlier, a POE item can take either a selected-response or a constructed- 
response format. In the selected-response format, students choose from a set of possible 
alternatives (based on known alternative mental models). In the constructed-response 
format, the student’s task is to write out (with justification) a prediction or an explana- 
tion. In the POE selected-response item above, students are asked to choose among a set 
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of predictions for whether the parts of the block will sink or float. They can choose an 
answer from several alternatives that include naive conceptions. This item could be ex- 
tended to further probe students’ mental models of buoyancy. Students could be asked to 
justify their answers by selecting from a number of possible explanations; again, these 
alternatives could include the correct explanation along with distractor explanations 
based on naive conceptions. Or, instead of using a multiple-choice format, students could 
be told that the full block sinks in water and be asked to write their prediction (with justi- 
fication) as to what will happen when the two parts are placed in water. Or, students 
could observe a simulation or video of what happens to the full block and the two parts 
and then be asked to explain what they have observed. 

Hands-On Performance Tasks 

Using hands-on performance tasks, students manipulate selected physical objects and try 
to solve a scientific problem involving the objects. These exercises, if carefully designed, 
can probe students’ abilities to combine their science knowledge with the investigative 
skills that reflect the nature of science and inquiry. In large-scale assessments such as 
NAEP, uniform administration must be ensured. In the past, this has been accomplished 
through the use of standardized performance assessment kits, with each exercise proc- 
tored and scored by trained personnel. Special accommodations may be necessary for 
some students. 

A particularly cogent criticism of most hands-on performance tasks administered in 
large-scale assessments is that, rather than tap into students’ ability to inquire into a prob- 
lem, typical performance assessments instead measure students’ ability to follow step-by- 
step instructions to arrive at the expected answer. Assessment developers are likely to 
create these “recipe” types of exercises because they must take into account the vast dif- 
ferences in students’ participation in science courses and their experiences. Given these 
differences, the absence of structure might produce unanticipated responses that could be 
problematic for the assessment either at the time the data are collected or when students’ 
performances are scored by raters. Although both the concern for structure to make large- 
scale assessment manageable and the criticism of highly structured performance tasks are 
well taken, there is evidence that valid performance exercises can be designed, devel- 
oped, administered, and scored without encountering major problems (e.g., Ruiz-Primo 
and Shavelson 1996b; Shavelson, Baxter, and Pine 1991). 

In designing hands-on performance tasks, it is important to keep the following considera- 
tions in mind: The degree to which students engage in some aspect of scientific inquiry 
depends on who selects the problem to be studied, who selects the procedures to be car- 
ried out in tackling the problem, and who selects the answer. The NAEP assessment 
should provide students with a challenging problem. However, students must have the 
opportunity to determine scientifically justifiable procedures for addressing the problem 
and arriving at a solution. Indeed, the problem to be solved involves setting forth proce- 
dures that manipulate the variable of interest, control extraneous variables, and provide 
solid data to be used in arguing for and justifying a problem solution. In addition to 
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allowing students to determine the procedures for carrying out the experiment, NAEP 
hands-on performance tasks should be “content rich” in that they require knowledge of 
science principles to carry them out. 

In brief, any hands-on performance task included in the NAEP assessment should present 
students with a concrete, well-contextualized task (problem, challenge); laboratory 
equipment and materials; and a response format that leaves the exercise process open. 
Students’ scores should be based on both the procedures created for carrying out the in- 
vestigation and the solution (Shavelson, Baxter, and Pine 1991). The assessment, then, 
should provide the problem that draws on science principles and practices (performance 
expectation of interest) and leave students free to design and carry out the exercise to ar- 
rive at an answer or solution. The following item, designed for fifth graders, is an exam- 
ple of such a task. 



Illustrative Item 

Students are asked to identify the contents of each of the six boxes (A-F) by using the 
batteries, bulbs, and wires they are given to complete a circuit. This task requires 
knowledge of series circuits but leaves problem-solving procedures up to the student. 
(See appendix C for more information about this task.) 







dI j 



Electric Myitcry Boici 



P4.ll, Using Scientific Inquiry 
Source: Shavelson, Baxter, and Pine 1991. 



Interactive Computer Tasks 

The 2009 NAEP Science Assessment should include some but not necessarily all of the 
following four types of ICTs: (1) information search and analysis, (2) empirical investi- 
gation, (3) simulation, and (4) concept mapping. This framework uses static screen shots 
to illustrate examples of ICTs; however, the screen shots represent only a small subset of 
the many screens students see when engaged in actual ICTs. 
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Information search and analysis items pose a scientific problem and ask students to query 
an information database to bring conceptual and empirical information to bear (through 
analysis) on the problem. Following is a screen shot from such an ICT developed for 
eighth graders. 



Illustrative Item 
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(See appendix C for more information about this task.) 



P8.16, Using Scientific Inquiry 
Source: Persky et al. 2005. 



Empirical investigation items put hands-on performance tasks on the computer and invite 
students to design and conduct a study to draw inferences and conclusions about a prob- 
lem. Whether the computer-simulated experiment assesses the same skills, knowledge, 
and understanding as a hands-on performance task has not been established; a special 
study will be conducted to address this question (see p. 117). Following is a screen shot 
from a computer version of the electric mysteries task (see p. 107 and appendix C for 
more information about this hands-on performance task). 
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Illustrative Item 




P4.ll, Using Scientific Inquiry 
Source: Shavelson, Baxter, and Pine 1991, p. 357. 
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Simulation items model systems (e.g., food webs), pose problems of prediction and ex- 
planation about changes in the system, and permit students to collect data and solve prob- 
lems in the system. The following screen shot comes from such a simulation ICT. 



Illustrative Item 



High school (grade 12) students are asked to identify locations appropriate for solar power 
generation. (See appendix C for a description of the full task.) To complete the task, students 
must: 

• Evaluate GIS map visualizations. 

• Compare and contrast visualizations of different types of data. 

• Use analytical extension to perform computations with visualization data. 

Following is an example of one of several tasks that a student completes: 

Your task is to identify two states that will have a high annual solar energy and will be able to 
generate the maximum amount of electricity from their solar panels. Name two states that you 
predict will have a good annual electrical yield. In the rest of this performance assessment, you 
will generate visualizations and calculate which states will generate the best annual electricity 
yield from solar panels. 



The following screen shot provides an example of a student response: 




This visualization shows that the states with the most annual solar energy 
are Arizona and Nevada. This data alone, however, does not completely 
justify the location of the panels, due to temperature issues. 

E12.10, Using Technological Design 



Source: Quellmalz et al. 2004. 
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Finally, concept mapping can be done by providing concept terms and asking students to 
build propositions on the computer by linking pairs of terms with arrows and words or 
phrases. Following is a screen shot of a completed concept map. 



Illustrative Item 

In this task, middle and high school students used a customized software program to 
create concept maps. Students received 18 environmental science terms and 7 link 
labels.* Students could drag and drop these concepts onto the grid space of the mapping 
program and add, erase, and link the items in their newly constructed maps. (See 
appendix C for more information about this task.) 




L12.5, Identifying Science Principles 
Source: Adapted from Herl et al. 1999. 
* Link labels should not be provided to students on the 2009 NAEP Science Assessment. See the specifica- 
tions for more information. 



Computers and other media provide potential solutions to a variety of practical challenges 
posed by complex assessment exercises. The logistical challenges of hands-on perform- 
ance tasks can be circumvented with computer simulation. Extensive databases can be 
used to assess students’ ability to select and evaluate information relevant to the situation 
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or problem they need to address. In addition, the difficulty of providing materials and 
training for complex tasks (such as concept maps) can be circumvented with computers. 
To avoid cheating and teaching to the concept map, concept terms can be randomly sam- 
pled for a particular map (Ruiz-Primo et al. 2001). 

ICTs should be used when the format offers advantages over other assessment modes. 
ICTs include (but are not limited to) testing student knowledge, skills, and abilities re- 
lated to the following situations: 

• For scientific phenomena that cannot easily be observed in real time, such as see- 
ing things in slow motion (e.g., the motion of a wave) or at a higher speed (e.g., 
erosion caused by a river). They are also useful when it is necessary to freeze ac- 
tion or replay. 

• For modeling scientific phenomena that are invisible to the naked eye (e.g., the 
movement of molecules in a gas). 

• For working safely in lab-like simulations that would otherwise be hazardous 
(e.g., using dangerous chemicals) or disorderly in an assessment situation. 

• For situations that require several repetitions of an experiment in limited 
assessment time while varying the parameters (e.g., rolling a ball down a slope 
while varying the mass, the angle of inclination, or the coefficient of friction of 
the surface). 

• For searching the Internet and resource documents that provide high-fidelity 
situations related to the actual world in which such performances are likely to be 
observed. 

• For manipulating objects in a simplistic manner, such as moving concept terms in 
a concept map. 

Extended constructed-response items, concept-mapping tasks, and simulated performance 
tasks are especially strong candidates for ICTs. In this way, the complex science under- 
standings and practices that need to be probed in the NAEP Science Assessment could be 
captured with less time, cost, and logistical challenges and with greater opportunity for 
divergent problem-solving tasks than has been the case in the past. 

Distribution of Items 



This section recommends appropriate item distributions. The three types of distribution, 
as measured by percentage of student response time at each grade level, are as follows: 

• Items by content area — Physical Science, Life Science, and Earth and Space 
Sciences 
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• Items by science practice — Identifying Science Principles, Using Science Princi- 
ples, Using Scientific Inquiry, and Using Technological Design 

• Items by type — Selected-response items and constructed-response items 

DISTRIBUTION OF ITEMS BY CONTENT AREA 

In the Overview of the Science Assessment section of this chapter, the distribution of 
items at each grade level by the three science content areas, as measured by percentage of 
student response time, was described as shown in exhibit 17. 



Exhibit 17. Distribution of items by content area and grade 





Grade 4 
(percentage of 
student response 
time) 


Grade 8 
(percentage of 
student response 
time) 


Grade 12* 
(percentage of 
student response 
time) 


Physical Science 


33.3 


30.0 


37.5 


Life Science 


33.3 


30.0 


37.5 


Earth and Space Sciences 


33.3 


40.0 


25.0 



* These recommendations are based on NAEP data regarding students’ course-taking patterns in 12th 
grade. If these patterns change substantially after 2009, the recommendations should be reconsidered. 



DISTRIBUTION OF ITEMS BY SCIENCE ERA CTICE 

The item distribution for the four science practices, as measured by percentage of student 
response time at each grade level, should have the following approximate allocations: 



Exhibit 18. Distribution of items by science practice and grade 





Grade 4 
(percentage of 
student response 
time) 


Grade 8 
(percentage of 
student response 
time) 


Grade 12 
(percentage of 
student response 
time) 


Identifying Science Principles 


30 


25 


20 


Using Science Principles 


30 


35 


40 


Using Scientific Inquiry 


30 


30 


30 


Using Technological Design 


10 


10 


10 



DISTRIBUTION OF ITEMS BY ITEM TYPE 

As measured by student response time, 50 percent of the assessment items at each grade 
level should be selected-response items and 50 percent should be constructed-response 
items (short constructed response, extended constructed response, concept-mapping 
tasks). Individual selected-response items and individual constructed-response items used 
within each of the item clusters, POE item sets, hands-on performance tasks, and ICTs 
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should be included in this distribution. If variation from this distribution becomes neces- 
sary as items are developed, preference should be given to constructed-response items. 



In regard to the combination item types, the NAEP Science Assessment should contain at 
least one of each of the following at each grade level: item clusters, POE item sets, 
hands-on performance tasks, and ICTs. In addition, it is highly recommended that each 
assessment include at least one concept-mapping task in grades 8 and 12. 

Hands-on performance tasks and ICTs are administered to a subset of the students sam- 
pled; the quantity for each task is specified in exhibit 19. 



Exhibit 19. Distribution of items by type of item and grade 





Grade 4 


Grade 8 


Grade 12 




(number of 


(number of 


(number of 




tasks) 


tasks) 


tasks) 


Hands-On Performance Task 


>1 


>1 


>1 


Interactive Computer Task 


>1 


>1 


>1 


Total 


>4 


>4 


>4 











In any grade, the number of ICTs should not exceed the number of hands-on performance 
tasks. No student will be administered both types of tasks. 

Students With Disabilities and English Language Learners 



Along with the increase in national and state testing, there is an increased demand for 
assessment systems to include all students (for example, those with disabilities and those 
learning English, many of whom have not been included in these systems in the past). As 
NAEP seeks to measure the educational progress of students in the nation’s classrooms, 
assessment developers will encounter challenges that require careful thought and consid- 
eration to the development of items that provide the most equitable context possible for 
all students. 

NAEP should strive to develop science assessments that allow the widest possible range 
of students to participate. In this way, the interpretation of scores of all participants leads 
to valid inferences about the levels of their performance as well as valid comparisons 
across states and with state assessments. All students should have the opportunity to 
demonstrate their knowledge of the concepts and ideas that the NAEP Science Assess- 
ment is intended to measure. 
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According to the National Research Council: 

Fairness, like validity, cannot be properly addressed as an afterthought 
once the test has been developed, administered, and used. It must be 
confronted throughout the interconnected phases of the testing process, 
from test design and development to administration, scoring, 
interpretation, and use (1999b, pp. 80-81). 

When assessments are first conceptualized, they must be thought of in the context of the 
entire population that will be assessed (American Educational Research Association, 
American Psychological Association, and National Council on Measurement in Educa- 
tion 1999; NRC 1999a; Thompson, Johnstone, and Thurlow 2002). NAEP assessments, 
as well as all large-scale assessments, must be responsive to growing demands: increased 
diversity, increased inclusion of all types of students in the general curriculum, and in- 
creased emphasis and commitment to serve and be accountable for all students. Assess- 
ments must measure the performance of students who have a wide range of abilities and 
skills, ensuring that those with diverse learning needs receive opportunities to demon- 
strate competence on the same content as all other students. 

Students with disabilities and English language learners each present challenges in how 
their science competencies can be assessed in a valid manner. Nevertheless, these groups 
share some commonalities, including considerable heterogeneity in each group regarding 
assessment needs. In addition: 

• Conceptual frameworks based on appropriate theories of language development 
and proficiency and of various forms of disabilities will be needed to build inclu- 
sive assessments. 

• A greater commitment to financial and human resources will be needed to de- 
velop, administer, and interpret performance on relevant tasks. 

Two general recommendations pertain to both groups in the context of good assessment 
design for all students: readability of written text and alignment to content statements. 

Students’ ability to read and respond to written text often determines successful perform- 
ance on assessments. Assessment items may pose an unfair disadvantage for some stu- 
dents if there is a heavy burden on reading skills when reading is not the goal of the as- 
sessment. Language that is both straightforward and concise and that uses everyday 
words to convey meaning is needed. The goal of ensuring that language has these charac- 
teristics is to improve the comprehensibility of written text and at the same time preserve 
the essence of its meaning. Using language that reduces the linguistic demands placed on 
students reduces the effect of reading skills and language proficiency on students’ science 
performance and assessment scores. More information on reading level is provided in the 
specifications. 
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Items on the NAEP Science Assessment must be aligned to the content statements and 
science practices with the same depth and breadth of coverage and the same cognitive 
demands as specified in the framework. Assessment design should emphasize accessibil- 
ity using different formats, technologies, designs, and accommodations to include as 
many students as possible. It must be clear from the outset that, to be equitable, assess- 
ments need to measure the achievement of all students on the same content and achieve- 
ment standards. 

To these ends, field tests should sample every type of student expected to participate in 
the final assessment administration, including students with a wide range of disabilities, 
English language learners, and students across racial, ethnic, and socioeconomic lines. 
Field testing NAEP items with a broad range of students will not only help determine 
whether items are unclear, misleading, or inaccessible for certain groups of students, but 
will also help ensure that assessment procedures are accessible to students when the 
NAEP Science Assessment is fully implemented. More information about these recom- 
mendations can be found in the specifications. 

NAEP strives to assess all students selected by its sampling process. Rigorous criteria are 
applied to minimize the number of English language learners and students with disabili- 
ties who are excluded from NAEP assessments. Participating students with special needs 
are permitted to use accommodations, as stated in current NAEP policy: 

All special-needs students may use the same accommodations in NAEP 
assessments that they use in their usual classroom testing unless the ac- 
commodation would make it impossible to measure the ability, skill, or 
proficiency being assessed, or the accommodation is not possible for the 
NAEP program to administer (NCES 2005a). 

See the specifications for more information on NAEP’s inclusion policy and permitted 
accommodations. 

Special Studies 



Special studies bearing on aspects of the 2009 NAEP Science Assessment are presented 
in this framework. Each study contributes to a further understanding of science assess- 
ment. 



Group 1 special studies have the highest priority: 

• “Exchangeability” of hands-on performance and interactive computer 
investigations 

• Impact of variation in item format and language demand on the performance of 
English language learners and students with disabilities 
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• Computer adaptive testing to assess the development of student understanding of 
Earth systems 

Group 2 special studies have lower priority: 

• Knowing what students know about technological design 

• Extended investigations by students 

The order in which studies are listed does not imply priority within group 1 or group 2. 
Group 1 special studies are described in the next section. Group 2 special studies are dis- 
cussed in appendix D. 

GROUP 1 SPECIAL STUDIES 

“Exchangeability” of Hands-on Performance and Interactive Computer 
Investigations 

Inquiry is at the heart of knowing and doing science. A fundamental aspect of inquiry is 
the design, conduct, and interpretation of empirical investigations to answer a question or 
test a hypothesis. Although a full assessment of inquiry is not possible on any test that is 
given on demand, hands-on performance investigations (HPIs) attempt to approximate 
this aspect of inquiry under time, space, cost, and logistical constraints. For this reason, 
HPIs have been a part of the NAEP Science Assessment since 1996. 

However, HPIs have been criticized as being expensive, logistically difficult, and too 
highly structured. On the other hand, ICTs or, in this case, interactive computer investiga- 
tions (ICIs), are logistically simpler, less expensive, and less structured. Therefore, the 
purpose of this study is to explore whether ICIs and HPIs are exchangeable. The question 
is not whether ICIs could replace HPIs either on NAEP or in the classroom — they should 
not. Even if these two approaches produce similar performances and scores, each affords 
somewhat different opportunities; simulations cannot be exchanged with actual practice. 
This is a question of assessment: can the cost and logistical challenges of HPIs be re- 
duced with the use of ICIs and still measure the same competencies with as much reli- 
ability and validity? Some research suggests that the two methods of assessing student 
inquiry are exchangeable to a fair degree (e.g., Pine, Baxter, and Shavelson 1993; Rosen- 
quist, Shavelson, and Ruiz-Primo 2000). However, further research is needed on several 
different investigations to provide a satisfactory answer for large-scale assessment. 

Specifically, this study would address the following research questions: 

• Does the choice of an ICI or HPI limit the questions that may be asked? 
Specifically, is there something of value in an HPI that cannot be asked if an ICI 
is administered? 

• Are scores on HPIs and ICIs equally reliable? 
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• Are scores on HPIs and ICIs of equal magnitude? 

• To what extent does performance on an HPI predict performance on an ICI in the 
same investigation? 

• Do scores on HPIs and ICIs correlate somewhat equally with scores on another 
measure of science inquiry or achievement? 

• Are similar thinking processes evoked by HPIs and ICIs? 

• Do the answers to these questions depend on individual differences among stu- 
dents (such as gender, English proficiency, race/ethnicity, socioeconomic status, 
and geographic location)? Variation in student access to computers is also an area 
of interest; this factor could be confused with other variables listed here. 



Impact of Variation in Item Format and Language Demand on the Performance of 
English Language Learners and Students With Disabilities 

English language learners and students with disabilities do not perform as well on stan- 
dardized achievement assessments as other students. Recent studies, for example, have 
pointed to a systematic relationship between the linguistic complexity of the assessment 
and the test scores of English language learners (e.g., Abedi 2003) and students with 
certain disabilities, such as those related to reading and information processing. Science 
assessments, with their heavy reliance on verbal skills, may exacerbate performance dis- 
parities. In cases where this relationship is demonstrable and test items use complex lan- 
guage, the differences become sources of measurement error and construct irrelevant 
variance; therefore, the nature of the assessment item must be addressed. Until this di- 
mension of the assessment item is more clearly understood, any interpretation of the 
performance of English language learners or students with disabilities on a content 
assessment is problematic; for example, language proficiency and science understanding 
cannot be separated. 

Preliminary results from several studies of scaffolded science assessments that are 
designed to minimize language complexity and provide alternative response modalities — 
including graphic organizers or drawn representations of the concepts — indicate that 
research is needed to clarify the relationship between language complexity, scaffolded 
assessment items, and the performance of English language learners and students with 
disabilities. 

Specifically, this study would address the following research questions: 

• Can the language complexity of a content-based assessment be systematically 
measured? 
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• Can content-based assessment items be designed to minimize the language de- 
mand while conserving the content information obtained? 

• If the content-based assessment contains a graphic response modality, do English 
language learners and students with disabilities demonstrate higher understanding 
of the content concept being assessed relative to more linguistically demanding 
response modalities? 

• When the content-based assessment with a graphic response option is also com- 
puter based, is there a further benefit in terms of content concept conservation and 
the performance of these students? 



Computer Adaptive Testing To Assess the Development of Student Understanding 
of Earth Systems 11 

A common critique of large-scale assessment is that its necessary reliance on easily 
scored, decontextualized, and simplified items has led to an impoverished range of poten- 
tial learning activities from which valid and reliable measures might be derived (Resnick 
and Resnick 1992). Attempts to find alternatives include the following: 

• The facets approach (Minstrell 1998), which posits a strong model of facets of 
student knowledge for certain science topic areas and uses coordinated sets of 
multiple-choice items to hone in on students’ particular conceptions and miscon- 
ceptions; and 

• The progress variable approach (Masters, Adams, and Wilson 1990), which posits 
a learning progression and uses item response theory to scale students’ responses 
to (typically open-ended) items to estimate the part of the learning progression in 
which students will most likely be located. 

This special study combines the strengths of each of these approaches to develop a new 
type of “branching” item that can be used to investigate the more complex types of 
knowledge structures and complex procedural steps involving contingencies such as 
those common in inquiry-related contexts, and yet maintain the efficiency of traditional 
multiple-choice testing. Specifically, the facets approach will contribute its strong knowl- 
edge structure and convenient scoring, and the progress variables approach will contrib- 
ute the interpretational framework of the learning progression and the flexible statistical 
modeling available through recent advances in item response modeling (De Boeck and 
Wilson 2004). Together, these approaches allow the use of item bundles (such as those 
shown in exhibit 20) to provide both the usual result in terms of student ability estimation 
as well as potentially more educationally informative results such as the prevalence of 
particular classes of misconceptions among the student body. 



1 1 Currently, a number of states are using computerized testing, consisting largely of translating traditional 
paper-and-pencil items into computer-based delivery systems. The study suggested here, as well as the first 
study on p. 1 14, makes more extensive use of the capabilities inherent in computer-based assessments. 
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Exhibit 20. Storyboard showing item design for a 
“branching” item bundle on ions and atoms (adapted from Scalise 2004) 

Matter Composition: Ions and Atoms Item Bundle 



1 . Lead-based paint contains Pb2+ ions and lead pipes are made up of Pb atoms 
The main difference between Pb2+ ion and Pb atom is: 

A. They are basically the same, (go to question 2) 

B.They have a different number of electrons, (go to question 3) 

CThey have a different number of protons (3) 

D.They are different but not in the ways described, (go to question 2 ) 

L I don't know ( 0 ) 



2. Choose the answer with which you most agree. 
Pb2+ Ion and Pb atom are the same except: 

A. Pb2+ has ionic bond and Pb has atomic bonds. (3) 

B. Pb2+ ion and Pb are similar but used differently. (1 ) 
CPb2+ is a liquid, Pb is a sold. (2) 

D. Pb2+ requires two Pb atoms. (2) 



3. Pick the best answer below: 

A. Pb2+ has 2 fewer electrons than Pb. 

(go to question 4) 

B. Pb2+ has a larger e- density cloud. (3) 
CPb2+ is positively charged so has 
2 extra valence electrons. (4) 



4. Electrons can be thought of as arranged in shells or orbitals. 
Select the ground state electron configuration for carbon: 

A. 1s22sl2p3 (7) 

B. 1s22s22p2 (go to question 5) — i 
C 1 s 22 s 22 p 1 ( 6 ) 

D. None of these (5) 



5. Certain main group elements exhibit multiple oxidation states, including Group 14 
of which carbon Is a member. It would seem that a possible oxidation state for carbon 
is +2, with two electrons available for bonding in half-filled 2p x and 2py orbitals. Based 
on valence bond theory, show an electron configuration justifying the formation 
of four bonds for carbon rather than two and describe the type of orbitals formed. 
EXPLAIN your answer as fully as possible. 

(All respondents who reach this screen will be scored as an 8 . 

Answers submitted here will be used to build distractors for this question.) 



Possible scores and 
score 0 

# paths 1 

parameters 871 



parameters for this 
1 2 

1 2 

§72 § 73 , 173 ! 



item bundle under iota model (assume constraint on 



3 

3 

874,1741,1742 



4 5 6 7 

1111 
§75 §76 §77 §78 



cases): 

8 

1 

§79 
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Specifically, this study would address the following research questions: 

• Can the “branching” item type be developed and delivered in a logistically effi- 
cient way for use in NAEP? 

• Can the information from sets of “branching” item bundles be used to provide re- 
liable, valid, and useful information on both overall student ability in science and 
the classification of students into educationally useful categories? 

The study would focus on a specific Earth system that is of practical and environmental 
significance, such as the biogeochemical carbon cycle. Understanding this system and 
related environmental issues (e.g., global climate change) requires connected understand- 
ings in the Physical, Life, and Earth and Space Sciences, many of which are characterized 
in the framework. For example, students who understand global warming understand how 
photosynthesis, cellular respiration, and fossil fuel combustion affect carbon dioxide con- 
centrations in the atmosphere. This connected understanding can be tracked as a learning 
progression. 
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Appendix A 

Steering Committee Guidelines 



The Framework Is Informed by the National Standards and 
Benchmarks 



The framework should reflect the nation’s best thinking in science instruction; therefore, 
it is guided by two national documents: National Science Education Standards (NRC 
1996) and Benchmarks for Science Literacy (AAAS 1993). Both documents were subject 
to extensive internal and external reviews during their development. 

Informed by the National Standards and Benchmarks, the framework should emphasize 
knowledge and use of science concepts, appropriate linking of science facts to concepts, 
relationships among concepts, and major themes unifying the sciences. The framework 
should also incorporate investigative skills. 



The Framework Reflects the Nature and Practice of Science 



The National Standards and Benchmarks include standards addressing science as inquiry, 
nature of science, history of science, and the designed world. The framework should em- 
phasize the importance of these aspects of science education and should include the ex- 
pectation that students will understand the nature and practice of science. Science is a 
self-correcting process; that is, a way of knowing in which theories are continually modi- 
fied and refined based on new research findings. Students should demonstrate the ability 
to accomplish the following: 

• make warranted inferences from evidence 

• use evidence to justify conclusions based on scientific investigations 

• demonstrate reasoning skills in the application of science content and in under- 
standing the connections between science concepts 

• exercise skepticism when evaluating, using, and discarding data 

• understand and use models to describe and do science 

• apply content knowledge and skills to solve problems as they occur in the natural 
world 
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• understand and apply knowledge of links and commonalities of science across 
fields 

The scientific disciplines are no longer practiced in isolation, and research that cuts 
across discipline boundaries is common. The framework should: 

• identify some of the science concepts and skills that cut across the assessed con- 
tent areas; 

• address science in both the natural and designed world; and 

• clearly define and identify commonalities and differences between “science” and 
“technology” or “technological design.” 

The framework should also address social and historical contexts, which are keys to un- 
derstanding how the scientific community has arrived at its current body of knowledge. 



The Framework Incorporates Key Attributes of Effective 
Assessment 



The framework should use assessment formats that are consistent with the objectives be- 
ing assessed. It should be guided by the best available research on assessment item design 
and delivery. 

The framework should take into account student diversity as reflected in gender, geo- 
graphic location, language proficiency, race/ethnicity, socioeconomic status, and disabil- 
ity condition. The assessment should be designed and written so it is accessible by the 
majority of students and to minimize the need for special accommodations for both stu- 
dents with disabilities and English language learners. Students with special needs should 
be provided accommodations so they can participate in the assessment. 

The framework should reflect knowledge about the acquisition of key science concepts 
over time, based on research about how students learn. The existing research findings 
should make clear, when possible, the progression of science knowledge across the grade 
levels. Concepts should be represented in a manner that reflects how students progress 
through a discipline and across disciplines. Assessment items should reflect students’ po- 
tential for applying concepts and more varied and complex situations over time. 

Critical content and skills should be articulated and assessed across grades 4, 8, and 12 
(vertically) and across the fields of science (horizontally) by creating items that are 
deliberately layered to achieve these goals. An example of measuring similar constructs 
within and across subjects is the progression of increasingly sophisticated understanding 
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about energy from elementary school to middle school to high school in the content areas 
of biology, chemistry. Earth science, and physics. 

The NAEP assessment should continue to use a variety of assessment formats, including 
well-constructed multiple-choice and open-ended items as well as performance tasks. In 
addition, multiple methods of assessment delivery should be considered, including the 
appropriate uses of digital-based technology. The framework should consider using digi- 
tal delivery systems for the assessment, including Web-based or CD-ROM formats. Em- 
bedded simulations that can represent scientific phenomena such as data, representations, 
and factors captured within laboratory experiments, along with the use of an adaptively 
designed series of assessment items, should also be considered. Advances in machine 
scoring of text should provide the opportunity for increased use of open-response format 
questions. The assessment format and delivery system employed should be accessible to 
the widest range of students. 

Each achievement level — Basic, Proficient, and Advanced — should include a range of 
items assessing various levels of cognitive knowledge that is broad enough to ensure each 
is measured with the same degree of accuracy. Descriptions of Basic, Proficient, and Ad- 
vanced must be as clear as possible. 



The Assessment Provides Data for Research 



NAEP assessment results are increasingly being used to review state student assessments 
and compare student achievement across states. The framework should address the im- 
portant uses of assessment data both to conduct research to better understand science 
learning and to improve science achievement. Data from the assessment should be col- 
lected in such a way as to provide information that accomplishes the following: 



• supplies details of the attributes (e.g., race/ethnicity, gender) of the students being 
assessed; 

• provides results by student gender, race/ethnicity, and socioeconomic level; 

• describes the academic preparation of the teachers whose students are being as- 
sessed; 

• describes the nature of the educational system of the students being assessed; 
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• relates the instructional delivery and materials, professional development of the 
teachers, and the learning environment to the results from the assessment; and 

• provides feedback to educators for improving science instruction and learning. 



The Specifications Document Is Closely Aligned With the 
Framework 



The connections among the framework, the specifications, and the assessment items 
should be transparent, have a consistent level of specificity, and be coherent. 

The specifications should be written with consistent detail across all fields, domains, and 
expectations of the framework: 

• The specifications should have a consistent structure across all areas. 

• The expected science knowledge that represents the target for assessment should 
be described in a clear and consistent format. The specifications should address 
content that reflects the standards and focuses on the significant information and 
knowledge that students should retain over time (e.g., principal ideas, fundamen- 
tal understandings). 

• The specifications should use verbs that describe the expected action to be taken 
in the assessment (e.g., identify, describe, evaluate, relate, analyze, demonstrate). 

• Expectations across the content areas should match in level of specificity and 
scope. 

• The specifications should follow the idea of learning trajectories. To assess over- 
arching concepts or themes, the assessment specifications should reflect a layered 
understanding of growth in knowledge of the concepts. 
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Appendix B 

NAEP Science Preliminary 
A chievement Level Descriptions 



Congress authorized the Governing Board to develop appropriate student achievement 
levels on NAEP. The achievement level descriptions are statements of what students 
should know and be able to do on NAEP at grades 4, 8, and 12. To fulfill its statutory re- 
sponsibility, the Governing Board developed a policy to guide the development of 
achievement levels for all NAEP subjects. It identified three levels of achievement to 
provide the public, educators, and policymakers with information on student performance 
on NAEP. These levels — Basic, Proficient, and Advanced — are used as a primary means 
of reporting NAEP results to describe “how good is good enough” at grades 4, 8, and 12. 

Exhibit B-l displays the Governing Board’s generic policy definitions for Basic, Profi- 
cient, and Advanced achievement that pertain to all NAEP subjects and grades. 



Exhibit B-l. Generic achievement level policy definitions for NAEP 



Achievement 

Level 


Policy Definition 


Advanced 


This level signifies superior performance. 


Proficient 


This level represents solid academic performance for each grade 
assessed. Students reaching this level have demonstrated 
competency over challenging subject matter, including subject- 
matter knowledge, application of such knowledge to real-world 
situations, and analytical skills appropriate to the subject matter. 


Basic 


This level denotes partial mastery of prerequisite knowledge and 
skills that are fundamental for proficient work at each grade. 



During the framework development process, the project committees are asked to develop 
preliminary achievement level descriptions, based on the generic policy definitions, to 
guide item development. The descriptions are designed to provide examples of what stu- 
dents performing at the Basic, Proficient, and Advanced achievement levels should know 
and be able to do in terms of the science content and practices identified in the frame- 
work; they are used to ensure that a broad range of items (covering Basic, Proficient, and 
Advanced levels) is developed at each grade level. The NAEP assessment development 
contractor and item writers are the intended audiences for the descriptions. 
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Exhibits B-2 to B-4 present the preliminary achievement level descriptions for grades 4, 

8, and 12 to clearly illustrate the science content and practices expected at each grade 
level. Planning committee members used their expert judgment to draft the preliminary 
descriptions and were partially guided by the following considerations: 

• Basic descriptions could be composed of both moderately challenging content and 
practice components. 

• Proficient descriptions could be composed of a mixture of highly and moderately 
challenging content and practice components. 

• Advanced descriptions could be composed of highly challenging science content 
and highly challenging components of a given science practice. 

The preliminary descriptions include only illustrative statements drawn from the frame- 
work’s science content and practices. The statements are not intended to represent the 
entire set of objectives from the content and practice dimensions, nor do the preliminary 
achievement level descriptions denote a sense of priority or importance based on the 
statements selected. In addition, the descriptions should not be interpreted as assessment 
items or as suggestions for assessment items. Although adjectives are not explicitly in- 
cluded in every statement, each was written with the expectation that student perform- 
ances would be “good” and “reasonable.” For example, the phrase “design an investiga- 
tion” implies “design an organized and logical investigation” and “propose a framework” 
implies “propose a rational framework.” 

After the assessment is administered, broadly representative panels engage in a standard- 
setting process to determine the achievement level cut scores on the NAEP scale. The cut 
scores represent the minimum score required for performance at each NAEP achievement 
level. A second outcome of this standard- setting process is a set of paragraphs, derived 
from the preliminary achievement level descriptions, to be used to report the NAEP 
science results to the general public and other audiences. For each grade level, the 
paragraphs describe what students should know and be able to do at the Basic, Proficient, 
and Advanced levels in terms of the science content and practices identified in the 
framework. 

More information on NAEP achievement levels is available at www.nagb.org. 
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Exhibit B-2. Grade 4 preliminary achievement level descriptions 





BASIC 


PROFICIENT 


ADVANCED 




Identifying Science Principles 


Identifying Science Principles 


Identifying Science Principles 


Physical 

Science 


Compare properties of solids and liquids. 


Describe the changes in physical properties 
that result when substances are heated and 
cooled. 


Identify properties of gases, liquids, and 
solids (e.g., whether they are made of one or 
more substances). 


Life 

Science 


Identify the foods that animals eat as sources 
of energy and building blocks for growth 
and repair. 


Describe life cycles of familiar plants and 
animals. 


Relate an organism’s survival with 
conditions in the environment that meet the 
organism’s basic needs. 


Earth and Space 
Sciences 


Describe changes in the apparent shape of 
the Moon over a month’s time. 


Describe the change in the apparent path of 
the Sun as viewed in the Northern 
Hemisphere between I une and December. 


Relate the changes in the location of sunrise 
with time of year. 




Using Science Principles 


Using Science Principles 


Using Science Principles 


Physical 

Science 


Describe what happens when a given 
amount of liquid is poured from a tall and 
narrow container into a broad and shallow 
container. 


Explain why a metal container filled with a 
hot liquid feels hotter to the touch than a 
foam container filled with the same hot 
liquid. 


Explain what happens to the gas in a helium 
balloon when it is punctured in a closed 
room. 


Life 

Science 


Describe how familiar animals meet their 
basic needs for food, air, water, and shelter. 


Predict how a change in a plant’s 
environment will affect the plant’s survival. 


Explain why animals need food and plants 
need nutrients. 


Earth and Space 
Sciences 


Explain why some rivers become muddy 
during or after heavy rains. 


Provide three examples of weathering and 
erosion. 


Relate periods of erosion to weather events 
or seasons. 
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Exhibit B-2. Grade 4 preliminary achievement level descriptions (cont.) 





BASIC 


PROFICIENT 


ADVANCED 




Using Scientific Inquiry 


Using Scientific Inquiry 


Using Scientific Inquiry 


Physical 

Science 


Order magnets by strength based on data 
describing the number of objects (e.g., paper 
clips) attracted to each of the magnets. 


Critique several proposed investigations 
comparing the heat produced by burning 
different quantities of wax. 


Design an investigation to demonstrate the 
relationship between the length of a 
vibrating string and the waves produced. 


Life 

Science 


Construct a bar graph from numerical data 
showing changes in the height of a plant 
over time. 


Select the best designed investigation from 
descriptions of several different ways to 
investigate the effects of light intensity on a 
plant. 


Design an investigation to demonstrate how 
a change in an environment changes the 
number of a familiar kind of animal in the 
environment. 


Earth and Space 
Sciences 


Identify a thermometer as a tool for 
measuring temperature. 


Construct a bar graph from data showing 
average monthly temperatures over a 12- 
month period. 


Design an investigation to relate weather 
data to the changes in seasons. 




Using Technological Design 


Using Technological Design 


Using Technological Design 


Physical 

Science 


Choose materials that are identified as good 
thermal insulators for use in containers to 
keep hot beverages from cooling off. 


Propose a method for testing materials to 
determine which ones are the best thermal 
insulators. 


Apply information about the thermal 
properties of materials to design a beverage 
container that keeps hot beverages hot and 
cold beverages cold for as long as possible. 


Life 

Science 


From three drawings of shelters for an 
animal, select the one that will meet most of 
the animal’s basic needs. 


Apply information about an animal’s basic 
needs to design a shelter for the animal. 


Analyze and explain how the design of a 
shelter or den, as constructed by an animal, 
meets the animal’s needs. 


Earth and Space 
Sciences 


Explain how to use the motion of the Sun to 
determine if it is morning or afternoon. 


From three different drawings, select the one 
that shows the best method of using the 
motion of the Sun to tell the approximate 
time of day. 


Explain how to safely observe and record 
the motion of the Sun during the day so as to 
construct a means of telling the approximate 
time of day. 
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Exhibit B-3. Grade 8 preliminary achievement level descriptions 





BASIC 


PROFICIENT 


ADVANCED 




Identifying Science Principles 


Identifying Science Principles 


Identifying Science Principles 


Physical 

Science 


Recognize that acids are a class of 
compounds that exhibit common chemical 
properties. 


Describe properties of acids such as color 
changes with acid/base indicators and the 
tendency to react with bases. 


Relate the properties of elements that form 
acids with their position in the periodic 
table. 


Life 

Science 


Identify producers and consumers as 
components of living systems. 


Describe the functions of consumers in 
ecological systems. 


Relate the functions of consumers to the 
energy flow in ecological systems. 


Earth and Space 
Sciences 


Describe the location of Earth in the solar 
system. 


Relate the phases of the Moon and the 
length of a day and a year to the motions of 
Earth, the Moon, and the Sun. 


Relate the force of gravity to the regular 
motion of bodies in the solar system. 




Using Science Principles 


Using Science Principles 


Using Science Principles 


Physical 

Science 


Explain the physical properties of solids, 
liquids, and gases using the idea that matter 
is composed of tiny particles in motion. 


Explain chemical properties of metals using 
the structure of atoms. 


Predict the properties of an element based 
on its position in the periodic table. 


Life 

Science 


Give examples of producers and consumers 
in aquatic ecosystems. 


Predict the effect of a reduction in the 
population of a predator species on the 
population of a species on which it preys. 


Evaluate an alternative explanation for 
patterns observed in an ecosystem’ s 
population data. 


Earth and Space 
Sciences 


Predict the effect of a reduction of the 
amount of light from the Sun reaching 
Earth. 


Explain how the tilt of Earth's rotation axis 
produces annual variation in the intensity of 
sunlight on the Earth's surface. 


Predict how changes in axial tilt would 
affect annual variations in the intensity of 
sunlight on the Earth's surface at different 
latitudes. 
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Exhibit B-3. Grade 8 preliminary achievement level descriptions (cont.) 





BASIC 


PROFICIENT 


ADVANCED 




Using Scientific Inquiry 


Using Scientific Inquiry 


Using Scientific Inquiry 


Physical 

Science 


Use physical and chemical properties to 
classify substances as metals or nonmetals. 


Select the best designed investigation from 
descriptions of several different ways to 
demonstrate that water does not change 
chemically when it changes state. 


Design an investigation to measure the 
temperature of water when it changes state 
(phase diagrams). 


Life 

Science 


Describe simple patterns in population data. 


Conduct a survey of an ecosystem’s 
population data and propose an explanation 
for patterns observed in the data. 


Design a survey of an ecosystem’s consumer 
populations and propose an explanation for 
patterns in the data based on energy flow 
through the system. 


Earth and Space 
Sciences 


Apply a framework for analyzing the 
constituent minerals and texture of a rock. 


Analyze properties of a rock formation to 
draw valid conclusions about the conditions 
under which it was formed. 


Propose a framework for classifying rocks 
based on constituent minerals, texture, and 
environment of formation. 




Using Technological Design 


Using Technological Design 


Using Technological Design 


Physical 

Science 


Interpret a distance versus time graph of 
two different types of parachutes (showing 
how far they have fallen in equal time 
intervals) to determine which one is better 
at slowing the jumper’s fall. 


Construct a graph based on a table showing 
how far two different types of parachutes 
have fallen in equal time intervals. 
Determine which parachute is better at 
slowing the jumper’s fall and describe the 
evidence that supports that determination. 


Design a method for measuring and 
comparing the effectiveness of two different 
types of parachutes to see which one is 
better at slowing the jumper's fall. 


Life 

Science 


Apply information about consumers and 
producers to critique the design of a self- 
sustaining terrarium. 


Apply information about consumers and 
producers to design a self-sustaining 
terrarium. 


Apply information about energy flow to 
critique a plan for managing the deer 
population in a forest ecosystem located 
near fields where corn is grown. 


Earth and Space 
Sciences 


Identify the possible effects on the 
environment of cutting down a forested 
area to establish a new field to grow corn. 


List the scientific tradeoffs that need to be 
taken into account when cutting down a 
forested area to establish a new field to 
grow corn. 


Taking into account the scientific tradeoffs 
involved in converting forested lands to 
fields for growing corn, propose actions that 
can be taken to reduce some of the negative 
environmental impacts while allowing for 
increased food production. 
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Exhibit B-4. Grade 12 preliminary achievement level descriptions 





BASIC 


PROFICIENT 


ADVANCED 




Identifying Science Principles 


Identifying Science Principles 


Identifying Science Principles 


Physical 

Science 


State the difference between exothermic and 
endothermic chemical reactions. 


Describe the increases in kinetic, rotational, 
and vibrational energy that result when a 
substance is heated. 


Explain the melting of a solid crystal in 
terms of the energy of the constituent 
molecules, ions, or atoms. 


Life 

Science 


State modern scientific ideas about 
evolution, such as natural selection and 
common descent. 


Describe fossil, anatomical, and molecular 
evidence for biological evolution. 


Make connections among the following 
related science principles: the potential of a 
species to increase its numbers, the genetic 
variability of its offspring, limitations on the 
resources required for life, and the ensuing 
selection of those organisms that are better 
able to survive and leave offspring. 


Earth and Space 
Sciences 


State the law of superposition in an 
undisturbed sequence of rock layers (that is, 
younger rock layers sit atop older rock 
layers below). 


Classify some geological processes as 
happening on a human time scale, such as 
earthquakes, and other processes occurring 
on a geological time scale, such as mountain 
building. 


Use index fossils and type sections to assign 
sequences of rocks to geological eras. 




Using Science Principles 


Using Science Principles 


Using Science Principles 


Physical 

Science 


Give an example of an element that has an 
isotope. 


Predict some common chemical reactions, 
given a choice of reactants (e.g., metals and 
nonmetals, acids and bases). 


Explain the difference between ionic and 
covalent bonding. 


Life 

Science 


Use antibiotic resistance as an example of 
principles of biological evolution. 


Predict the spread of infectious disease 
based on basic concepts of evolution. 


Use basic concepts of evolution to explain 
antibiotic resistance and invasive species. 


Earth and Space 
Sciences 


Explain the process of mountain building 
using the theory of plate tectonics. 


Explain the location of deep sea trenches as 
an outcome of geologic processes. 


Propose geologic processes that explain 
structures found on a geologic map. 
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Exhibit B-4. Grade 12 preliminary achievement level descriptions (cont.) 





BASIC 


PROFICIENT 


ADVANCED 




Using Scientific Inquiry 


Using Scientific Inquiry 


Using Scientific Inquiry 


Physical 

Science 


Describe patterns of physical and chemical 
properties within rows of the periodic table. 


Select the best designed demonstration 
among descriptions of how to illustrate the 
translational, vibrational, and rotational 
motion of molecules. 


Design an investigation to determine the 
effect of surface area on evaporation rate. 


Life 

Science 


Use information contained in a food web to 
illustrate energy conservation in an 
ecosystem. 


Select the best designed investigation among 
descriptions of ways to determine the effect 
of fertilizers on the growth of plants. 


Design a strategy for estimating the quantity 
of plant material required to produce a 
kilogram of beef. 


Earth and Space 
Sciences 


Analyze photographs of rock layers to 
determine the order in which the layers were 
deposited. 


Describe how radioactive dating is used to 
estimate the age of rock formations and 
sequence data to provide relative age 
information. 


Integrate fossil, stratigraphic, structural, and 
rock-type information to identify past spatial 
relationships between environments. 




Using Technological Design 


Using Technological Design 


Using Technological Design 


Physical 

Science 


A railway company wants to purchase the 
fastest bullet train on the market. Select the 
fastest train based on data about distance 
traveled by each of two trains and the time it 
took to travel that far. 


A railway company wants to purchase the 
fastest bullet train on the market. Design a 
test of two trains (consider what is to be 
measured, instruments to be used, and how 
data are to be collected). 


A railway company wants to purchase the 
fastest bullet train on the market. List 
science-based criteria and constraints that 
might be used to select the best train and 
propose a method to compare two trains on 
these criteria and constraints. 


Life 

Science 


Given a description of plants and animals in 
a stable ecosystem, identify from a list of 
possibilities the changes that might occur if 
people migrate into the area. 


Given a description of plants and animals in 
a stable ecosystem, list some of the changes 
that might occur if people migrate into the 
area and describe how those changes could 
affect the entire food web. 


Given a description of plants and animals in 
a stable ecosystem, list some of the changes 
that might occur if people migrate into the 
area and describe what the new residents 
could do to limit those changes and avoid 
affecting the entire food web. 


Earth and Space 
Sciences 


Given data on increasing levels of carbon 
dioxide in the atmosphere, select from a list 
of possible anticipated effects of this carbon 
dioxide buildup. 


Given data on increasing levels of carbon 
dioxide in the atmosphere, list both positive 
and negative effects of this carbon dioxide 
buildup. 


Given data on increasing levels of carbon 
dioxide in the atmosphere, list positive and 
negative effects of this buildup and science- 
based actions people might take to reduce or 
prepare for any negative effects. 
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Appendix C 



Sample Items and Scoring Guides 



Item Source: TIMSS 1999, Grade 8 (framework, p. 65). 




Look at the food web above. If the com crop failed one year, 
what would most likely happen to the robin population? 
Explain your answer. 
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Note: A correct response must include a feasible explanation directly relating the predicted change 
in robin population to the effect of corn crop failure on prey/predator relationships indicated 
in the food web. Responses do not have to use the specific terms decrease, increase, and 
same, as long as the explanation is clear with respect to the effect on the robin population. If 
more than one effect is given, assign the code corresponding to the first correct explanation. 



Code Response Item: S022141 

Correct Response 



10 


Robin population may decrease. Explanation based on predators (snakes/hawks) eating 
more robins if mice die. 

Examples: Goes down. The mice would stan’e and the snake would eat the robins. 
There would be less robins because the mouse population would 
decrease and the snakes (and/or hctwks) would eat more of the robins. 


11 


Robin population may increase. Explanation based on predators (snakes/hawks) dying 
due to lack of food (mice). 

Examples: It would go up because the snakes die if the mouse starves. 

There could be more robins because there are fewer snakes (and or 
hawks) to eat them. 


12 


Robin population would stay the same with a feasible explanation. 

Example: It Mould not change because the mouse Mould find other grain to eat 

so the snake M ould be unaffected. 


19 


Other acceptable explanation. 




Incorrect Response 


70 


Robin population would decrease. Incorrect explanation based on robins starving if 
snakes die (confuses prey/predator relationship). 

Examples: Decreases because there are less snakes to eat. 

When corn dies, then snakes, then robins starve. 


71 


Robin population would decrease. Incorrect explanation based on the robin needing 
corn to survive. 

Example: Decrease because they need the corn. 


72 


Robin population would stay the same. Incorrect explanation based on the robins not 
needing corn to survive or not being connected to corn in the food web. (Does not 
consider the effect of predators.) 

Examples: Nothing because the robin only eats insects. 

Nothing would happen. The corn is on a different chain in the food web. 


73 


Mentions only that the whole food web will be upset and/or all the animals will die. 
Example: The whole food web Mould erupt and everything Mould die. 


79 


Other incorrect (including crossed out/erased, stray marks, illegible, or off task). 




Nonresponse 


99 


BLANK 
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Item Source: Colorado Department of Education 2002, Grade 8 (framework, p. 68). 



The graph below shows the distance traveled over time by a student 
walking down a hall. Use the information shown on the graph to solve 
questions 7 and 8. 



Distance Traveled Over Time 




7. During which time interval was the student moving the fastest? 

O A 
O B 
O C 
O D 

Key: D 

8. What was the average speed of the student from 0 seconds to 5 seconds? 

Average speed: . 

Scoring: Average speed is 2 m/s. (The student traveled 10 meters in 5 seconds.) 
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Item Source: Quellmalz et al. 2004 (framework, p. 71). 



Lynx/Hare Task 

This is an interactive computer task in which students are expected to conduct a scientific 
investigation regarding the question of whether or not lynx should be introduced into a 
national park in order to reduce the abiding overpopulation of hares. Students are directed 
to complete six modules that use different computer programs to determine the best solu- 
tion for the proposed question: 

• Module 1 asks the student to access, organize, analyze, and interpret data they are 
given about the populations of hares over the past 4 years. For this module, stu- 
dents can use word processor, spreadsheet, or presentation software. 

• Module 2 asks the student to determine a better way to analyze and display some 
disorganized data that show how many lynx and hares were present each year 
over the past 25 years. 

• Module 3 first asks the student to submit a Web search that will give insight into 
the relationship between the lynx and hare populations. It subsequently asks the 
student to critically evaluate the relevance of several supplied Web searches. 

• Module 4 asks the student to collect information on specific questions regarding 
the lynx/hare question, to take notes on the information given on the websites, and 
to include citations for each site. 

• Module 5 asks the student to use a modeling program to predict the results of add- 
ing more lynx to the parks through viewing population trends over several years. 
Students are then asked questions based on what they have observed in the model- 
ing tool regarding increases or decreases in the hare population if lynx are added 
to the park or not added to the park. 

• Module 6 asks the student to create a presentation with word processor or presen- 
tation software to communicate the problem, the findings, and any recommenda- 
tions that resulted from the newly completed research. 

For this task, students receive scores for inquiry skills and technology use, and for the 
appropriate use of concepts within their explanations and recommendations. 

For more information, see http://ipat.sri.com/tasks/pred_prey/subtasks/taskstud.html. 
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Item Source: Washington Assessment of Student Learning 2004, Grade 8 (framework, p. 75). 



Occasionally, a fire will destroy a forest, burning down trees and pushing wildlife out of their 
forest homes. However, the forest will grow back. Eventually, through the process of forest suc- 
cession as shown below, short grasses and flowers begin to grow and animals make new homes. 




Forest Succession 



Short 

grasses 



Tall 

grasses 



Shrubs 



Time 



Small 

trees 



Tall 

trees 



Over time, shrubs and trees begin to grow. The forest returns to a lush habitat for the 
wildlife listed in the chart below. 



Forest Wildlife 



Ground dwelling 


Worms, beetles 


Reptiles and amphibians 


American toads, wood frogs, snakes. Eastern box turtles 


Small animals 


Squirrels, chipmunks 


Medium to large animals 


Opossums, raccoons, white-tailed deer, black bears 


Airborne 


Butterflies, moths, bees, wild turkeys, red-tailed hawks, 
bald eagles 



A power company owns paid of a forest that was destroyed by a fire. The forest could 
take decades to rebuild on its own. The company’s department of environmental stud- 
ies suggests planting new trees to help the forest rebuild. 

Using the information in the scenario: 

• Explain how planting trees could benefit the natural ecosystem. 

• Explain how planting trees could harm the natural ecosystem. 
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Scoring Rubric 

2-point response: The response demonstrates that the student can analyze how human societies’ 
use of natural resources affects the quality of life and the health of ecosystems. 

The student explains one reasonable way in which planting the trees could benefit the 
natural ecosystem. 

AND 

The student explains one reasonable way in which planting the trees could harm the 
natural ecosystem. 

Example responses: 

Benefits of planting the trees include but are not limited to: 

• providing a habitat for animals; 

• providing a canopy, which would help prevent soil erosion; 

• creating root systems, which would anchor soil in place; and 

• creating shade, which would help maintain sunlight levels and inhibit the intro- 
duction of nonnative plant species. 

Harms of planting the trees include but are not limited to: 

• disrupting the natural flow of animals reentering the forest; 

• inhibiting the growth of other plants; 

• decreasing the diversity of tree species growing in the forest; and 

• introducing foreign species into an area, which may affect native species of 
plants and animals. 

1-point response: The response demonstrates that the student can partially analyze how human 
societies’ use of natural resources affects the quality of life and health of ecosystems. 

The student explains one reasonable in which planting trees could benefit the 
environment. 

OR 

The student explains one reasonable way in which planting trees could harm the 
environment. 

0-point response: The response demonstrates that the student can perform little or no analysis of 
how human societies’ use of natural resources affects the quality of life and the health of 
ecosystems. 

Note: Benefits/harms to the natural ecosystem that only relate to humans will not be cred- 
ited score points. 
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Annotated example of a 2-point response: 

Explanation of how planting trees could benefit the natural ecosystem: 

The ecosystem will rebuild more quickly, and that would give the 
animals a new habitat more quickly. 

Explanation of how planting trees could harm the natural ecosystem: 

1 1 could disrupt the natural order of the ecosystem. I f some 
smaller plants don't get a chance to grow first, the trees might 
push them out. 

Annotation: 

The response demonstrates that the student can analyze how human societies’ use of natural re- 
sources affects the quality of life and health of ecosystems. 

The student explains one reasonable way in which planting the trees could benefit the natural 
ecosystem, “The ecosystem will rebuild more quickly, and that would give the animals a new 
habitat more quickly.” (1 point) The student explains one reasonable way in which planting the 
frees could harm the natural ecosystem, “If some smaller plants don’t get a chance to grow first, 
the trees might push them out.” (1 point) 

Annotated example of a 1-point response: 

Explanation of how planting trees could benefit the natural ecosystem: 

More plants and trees would mean giving off more oxygen, more shade, and 
more food for forest animals. 

Explanation of how planting trees could harm the natural ecosystem: 

There could be a very big storm and trees could fall on houses. 

Annotation: 

The response demonstrates that the student can partially analyze how human societies’ use of 
natural resources affects the quality of life and the health of ecosystems. 

The student explains one reasonable way in which planting the trees could benefit the natural 
ecosystem: “More food for forest animals.” (1 point) The student explains one way in which 
planting the frees could harm the human structure: “Trees falling on houses” but not the natural 
ecosystem. (0 points) 
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Annotated example of a 0-point response: 



Explanation of how planting trees could benefit the natural ecosystem: 

have more trees 

Explanation of how planting trees could harm the natural ecosystem: 

half the plants will die 

Annotation: 

The response demonstrates that the student can perform little to no analysis of how human socie- 
ties’ use of natural resources affects the quality of life and the health of ecosystems. 

The student states one factor, “have more frees,” but does not explain one reasonable way in 
which planting the trees could benefit the natural ecosystem. (0 points) 

The student states a possible harm, “half the plants will die,” but does not explain how planting 
the trees could cause this harm to the natural ecosystem. (0 points) 



Item Source: NAEP 1996, Grade 8 (framework, p. 83). 

A space station is to be located between Earth and the Moon at the place where Earth’s gravita- 
tional pull is equal to the Moon’s gravitational pull. On the diagram below, circle the letter indi- 
cating the approximate location of the space station. 




Scoring Rationale: Student demonstrates the ability to explain the role of gravity in a manmade 
satellite and relates the force of gravity to the mass (size) of the object pulling it. 

3 = Complete - Student circles point C and gives a correct explanation that gravitational pull 
depends on mass and distance; thus, the station must be closer to the Moon because the 
Moon’s mass is less than that of Earth. 

2 = Partial - Student circles point C and explains that the Moon has less gravity than Earth 
but does not link it to mass. 

1 = Unsatisfactory/Incorrect - Student circles A, B, or C and gives an incorrect explanation 
or no explanation. 
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Sample Student Responses 

Complete (Level 3) 

Explain your answer. 

PniMt C besjxjAAPs tk& zavths a, s ironies 
gnu ri tixtio n al puJ&JbjeMJdAb.af.tiz s ize, so . t k& 
station would have, to LocateAs nearer to tke, 
moon to equals jndU 



Partial (Level 2) 

Explain your answer. 

tUj>, Favtks Iiax a , anjitpy arainhiti/maL null 
tfuin the, moon so Lt needs to be, closer to ike, 
moon. 



Unsatisfactory /Incorrect (Level 1) 



Explain your answer. 

It .mulA.h&. r ight Ln. Middle,.. < h& to 

grajtitixtional Jocus 
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Item Source: TIMSS 2003, Grade 4 (framework, p. 96). 



On a hot, humid day the air contains a lot of water vapor. What happens to the water vapor in the 
air when the air becomes very cold? 

Note: Priority should be given to code 10. If a response mentions condensation or freezing, 

code 10 should be the response even if other correct codes apply. Responses that mention 
ONLY that the water vapor becomes cold or rises without any mention of a change of 
state (explicitly or implicitly) are scored as incorrect (code 70 or 71). 



Code 


Response 


Item: S031382 




Correct Response 


10 


Refers to condensation or freezing (or equivalent). 

Examples: It freezes. 

It condenses. 

Condensation. 

It condenses and turns into rain. 


11 


Mentions cloud formation or a form of precipitation (e.g., rain, snow, fog, etc.). 
Examples: The water vapor changes to rain. 

It changes to snow. 

Water vapor turns into clouds. 

It rises into the clouds and becomes rain droplets. 

It turns foggy. 

It rains. 


19 


Other correct 

Examples: It falls to the ground. 




Incorrect Response 


70 


Mentions only that the water becomes cold. [No mention of a change of state or precipitation.] 
Examples: The water vapor becomes cold. 

Its temperature drops. 


71 


Mentions only that water vapor rises (or similar). [No mention of condensation or precipitation.] 
Example: The water vapor will rise on a hot day. 


79 


Other incorrect (including crossed out/erased, stray marks, illegible, or off task). 
Example: It disappears. 




Nonresponse 


99 


Blank 
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Item Source: New Standards Spring Field Test 1999 for High School (framework, p. 96). 



The biosphere (living organisms), lithosphere (rocks and soils of Earth’s crust), and atmosphere 
arc all involved in the cycling of carbon atoms. Describe the role each plays in the carbon cycle. 



Scoring guide: 



Score Point 


Description 


4 


The response describes the cycling of carbon among all three reservoirs. The 
response is complete and detailed, showing evidence of logical reasoning. There 
is no evidence of misconceptions. 


3 


The response describes the cycling of carbon between two of the reservoirs. The 
response may contain omissions or minor errors. 


2 


The response describes the presence of carbon compounds in two or more 
reservoirs but may not link the reservoirs. The response may contain errors or 
misconceptions. 


1 


The response is largely incomplete, lacks detail, and contains errors of fact and 
reasoning. 


0 


The response may contain words from the question but does not add any 
information that might answer the question. 


Off topic 


Off-topic response. 


Blank 


No marks were made in the student response section. 



Background information for appropriate response: 

Carbon forms a variety of compounds. Carbon monoxide (CO) and carbon dioxide (CO 2 ) are car- 
bon compounds that exist as gases in the atmosphere. Organic carbon compounds come in a wide 
variety of forms in living organisms: sugars, carbohydrates, cellulose, starches, collagen, chitin, 
amino acids, proteins, lipids, and nucleic acids. These organic compounds arc also to be found in 
soil due to decaying and decomposition. The carbonate ion (C0 2 2 ) combines with different posi- 
tive ions to form carbonate compounds found in rocks and soil. 

Producers (green plants) take carbon dioxide from the air and convert it to sugar, using solar en- 
ergy (photosynthesis). Additional nutrients (including carbonate ions from the soil) are taken into 
the plant via its root system. These nutrients are rearranged and turned into the wide array of car- 
bon compounds listed above, along with the sugar - compounds formed via photosynthesis. As 
consumers eat producers, the carbon compounds are broken down and rearranged into carbon 
compounds used by the consumer organism. 

When any living organism dies, the carbon compounds are broken down and returned to the soil 
by decomposer organisms. Huge sediment layers of dead marine organisms, which once had chi- 
tin cell walls or shells, compress under the weight of continuing sediments and the ocean. Uplift- 
ing of these layers forms sedimentary cliffs and rocks. Weathering and erosion continue to 
change the carbon compounds into forms that will again be incorporated into living organisms. 

Producer, consumer, and decomposer organisms harness the chemical bond energy stored in food 
via cellular respiration. Cellular respiration releases carbon dioxide back into the atmosphere. 

The score is based on evidence of: geochemical cycling of carbon. 
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Item Source: Shavelson, Baxter, and Pine 1991 (framework, pp. 102, 104). 

Electric Mysteries 

The following is a brief description of two warmup tasks: 

1. Students are asked to connect one battery, one bulb, and wires so the bulb lights. They 
are then asked to draw a picture of this simple circuit. 

2. Given mystery box “?,” students are asked to identify whether it contains a battery or a 
wire. They are told they can determine the contents of the mystery box by connecting it 
in a circuit with a bulb. 

The following is an excerpt from the main task instructions given to students: 

Find out what is in the six mystery boxes A, B, C, D, E, and F. They have five different things 
inside, shown below. Two of the boxes will have the same thing. All of the others will have 
something different inside. 

[The five options — two batteries, a wire, a bulb, a battery and a bulb, nothing at all — are pre- 
sented in words and drawings. Drawings are not provided here.] 

For each box, connect it in a circuit to help you figure out what is inside. You can use your bulbs, 
batteries, and wires in any way you like. 

When you find out what is in a box, fill in the spaces on the following pages. 

The following is an example of the student response format: 



Box A: HAS 



INSIDE. 



Draw a picture of the circuit that told you what was inside Box A: 




The following is a brief description of the scoring system: 

For each of the six boxes (A-F), students’ responses are scored on two components: (1) identifi- 
cation of the contents of the box and (2) the circuit used to make the conclusion. For each box, if 
both components are correct, the student receives 1 point; if one or both components are incor- 
rect, the student receives 0 points. Total maximum score is 6 points. 
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Item Source: Persky et al. 2005 (framework, p. 103). 



TRE Web Search Task 

In this task, students arc asked to use a search engine to determine why scientists use large helium 
gas balloons to explore outer space and the atmosphere instead of using satellites, rockets, or 
other such tools. One open-ended question and several multiple-choice questions arc presented to 
students. Students are scored on how well they performed the search, the quality of the book- 
marked pages, and how well the questions were answered. 

A complex scoring framework is used to assess students’ proficiency on the task. A “student 
model” is created that describes the “theory” of how different skills are linked together. For this 
task, there arc five component skills: problem solving in technology-rich environments, computer 
skill, scientific inquiry skill, exploration, and synthesis. An “evidence model” is created that de- 
scribes how the students’ responses are connected to each of these skills. Once students begin the 
task, every action is recorded and connected to one or more skills in the student model. Then, a 
three-step process is used to evaluate this record. The first step is feature extraction, which shows 
what action the student took, when the action was taken, and any value that was associated with 
that action. The second step is feature evaluation, which gives the scores for the actions taken 
based on developed rules that show the best way to complete the task. The third step is evidence 
accumulation, which systematically combines responses into summary scores that detail the in- 
ferences that can be made from the students’ responses. 

This task incorporates a multifaceted evaluation process that can uncover the many skills in- 
volved in a single task or module. Thus, this module can provide a more comprehensive evalua- 
tion of students’ skills and aptitudes than traditional test questions. 

Item Source: Quellmalz et al. 2004 (framework, p. 105). 

Solar Power Task 

This is an interactive computer task in which students act as energy consultants to identify which 
two states will generate the most amount of electricity from photovoltaic cells. There are four 
modules in which students use a series of map visualizations and other data to reach and to pre- 
sent their final conclusions. Ultimately, students are asked to create a presentation to one state, 
recommending that the state apply for federal funds marked for solar energy use. 

Module 1 asks the student to review and apply background information on the conditions that 
both optimize and reduce solar energy production. They are also asked to conduct simple 
analysis using the ArcView map visualization program. 

Module 2 asks the student to explore several datasets to identify states with high incoming solar 
radiation and to manipulate the ArcView program and its map calculator tool. The student 
uses the ArcView program to generate visualizations and to calculate which states will gener- 
ate the best monthly and annual electricity yields from solar panels. 

Module 3 asks the student to determine what other data may be necessary to create the most com- 
pelling recommendation to the states. 
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Module 4 asks the student to create a presentation to one of the two states determined to have the 
highest capacity for solar energy production. The student uses the newly gathered data to 
support the recommendation to the state. 

Both generic and item-specific rubrics are used for scoring; they include scores for content (math 
and science), use of problem-solving or inquiry strategies (planning and thorough communica- 
tion), and use of technological tools. 

For more information, see http://ipat.sri.com/tasks/solarpowcr/subtasks/solar_tasks.html. 



Item Source: Adapted from Herl et al. 1999 (framework, p. 106). 

Concept-Mapping Task 

In this task, students use a custom software program to create a concept, or knowledge, map. Stu- 
dents are given 18 environmental science terms (atmosphere, bacteria, carbon dioxide, climate, 
consumer, decomposition, evaporation, food chain, greenhouse gases, nutrients, oceans, oxygen, 
photosynthesis, producer, respiration, sunlight, waste, and water cycle) and 7 link labels (causes, 
influences, paid of, produces, requires, used for, and uses). Students can then drag and drop these 
concepts onto the grid space of the mapping program and add, erase, and link the items in their 
newly constructed maps. The concept maps arc scored based on semantic content, organizational 
structure, number of terms used, and number of links made. 

Additionally, students explore a simulated World Wide Web space, which allows them to search 
for relevant environmental science information to improve their concept maps. Students can 
bookmark Web pages they believe are helpful in constructing their concept maps. This portion of 
the task is scored based on relevant information found, the hypertext links that were selected 
(browsing), keyword searching, and the accessing of three or more highly relevant Web pages for 
a single concept (focused browsing). 

While performing the concept-mapping task, students are able to access real-time feedback, 
which compares students’ maps to experts’ maps and gives corresponding feedback about which 
items are correct and which need improvement. A score is also assigned to how often a student 
accessed feedback, a measure of monitoring one’s learning. 

This computer-based assessment provides a detailed view into both the student’s ultimate per- 
formance and the steps of the thought processes that were employed to generate the ultimate 
product or answer (in this case, the concept map). 
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Appendix D 

Group 2 Small-Scale Special Studies 



Knowing What Students Know About Technouogicau Design 



Knowledge about technology and the technological design process are prominent in both 
the National Standards and Benchmarks . The National Standards state, “Although these 
are science education standards, the relationship between science and technology is so 
close that any presentation of science without developing an understanding of technology 
would portray an inaccurate picture of science” (NRC 1996, p. 190). Despite taking some 
time for schools to include technology in the curriculum for all students, there is growing 
recognition that technology should be an important component. An increasing number of 
states include technology in their standards — 30 states in 2001 and 38 states by 2004 
(Meade and Dugger 2004). Consequently, NAEP is including the process of technologi- 
cal design as parallel to, but with less emphasis than, the process of scientific inquiry. 

Because relatively few questions on NAEP will probe the practice of Using Technologi- 
cal Design, this study proposes the development of an additional set of questions to probe 
students’ understanding of this practice indepth. 

Specifically, this study would address the following research question: 

What do students know about technological design in the contexts of agricultural 
technologies, energy generation technologies, and technologies related to Earth mate- 
rials and resources? 

Extended Investigations by Students 



National and local science education standards emphasize scientific inquiry. In many 
states, this goal requires student engagement in projects that can take days, weeks, and 
even months as they undertake genuine investigations. Important outcomes of these pro- 
jects include a range of skills that are a crucial feature of high-quality science education 
but that cannot be assessed adequately in a 50-minute assessment (NRC 2005). They in- 
clude, for example, gauging the quality of students’ (a) reasoning while framing their re- 
search questions; (b) planning for data collection and the execution of that plan; (c) abil- 
ity to meet unpredictable challenges that arise during any actual, ongoing scientific inves- 
tigation; (d) persistence in seeking productive explanations for their observations and re- 
vising plans for the investigation; (e) lines of argument in deciding how to alter their ex- 
perimental approach in the light of new evidence; (f) engagement with fellow students 
and/or the teacher in interpreting an observation or result and deciding what to do about 



NAEP 2009 Science Framework 
149 






it; and (g) deliberations when settling on the defensible conclusions that might be drawn 
from their work. 



In many countries, teachers are expected to assess students’ work during extended pro- 
jects. Often their judgments of student achievement are made during ongoing classroom 
activities that are part of the regular curriculum. The teachers’ assessments are incorpo- 
rated into an overall score that also includes results of the short, timed tests. In some 
countries, a defined percentage of the total score is based on teachers’ judgments about 
achievement associated with investigative projects. 

This study, then, might include both a national sample of students and an exploration of 
what other countries do under similar circumstances. Specifically, the study would ad- 
dress the following research questions: 

• What methods can be or have been developed to assess student achievement with 
respect to the ability to conduct extended scientific investigations? 

• To what extent are shorter investigations interchangeable for the extended inves- 
tigation and to what extent are they not interchangeable? 
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